Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/src/extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,11 @@ double floating point numbers.
Extremely large or small values can still overflow or underflow to infinity or zero,
see note in `seq`.

## `tr`

Just like on FreeBSD and macOS, `tr` accepts `-u` to guarantee that any output
is unbuffered: each chunk is written as soon as it has been read and translated.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If unbuffered I/O is actually default of GNU tr, we can just say:
Noop. Existing for FreeBSD and macOS compatibility.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I missed comment about Windows.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, GNU tr isn't unbuffered by default either, it writes through stdio. With GNU tr 9.12, { printf abc; sleep 2; printf 'def\n'; } | tr a-z A-Z | cat only prints ABC after the 2 seconds, while stdbuf -o0 tr prints it right away. On Unix it's only our tr that writes every chunk immediately, since #14897.

This comment was marked as low quality.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry. Your doc is correct.


## `unexpand`

GNU `unexpand` provides `--first-only` to convert only leading sequences of blanks. We support a
Expand Down
1 change: 1 addition & 0 deletions src/uu/tr/locales/en-US.ftl
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ tr-help-complement = use the complement of SET1
tr-help-delete = delete characters in SET1, do not translate
tr-help-squeeze = replace each sequence of a repeated character that is listed in the last specified SET, with a single occurrence of that character
tr-help-truncate-set1 = first truncate SET1 to length of SET2
tr-help-unbuffered = guarantee that any output is unbuffered

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tr-help-unbuffered = guarantee that any output is unbuffered
tr-help-unbuffered = do not buffer output

We should be same with GNU, but we don't need to be same with BSD.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be changed in src/uu/tr/locales/fr-FR.ftl as well

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9853afb, and updated the French string to match.


# Error messages
tr-error-missing-operand = missing operand
Expand Down
1 change: 1 addition & 0 deletions src/uu/tr/locales/fr-FR.ftl
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ tr-help-complement = utiliser le complément d'ENSEMBLE1
tr-help-delete = supprimer les caractères dans ENSEMBLE1, ne pas traduire
tr-help-squeeze = remplacer chaque séquence d'un caractère répété qui est listé dans le dernier ENSEMBLE spécifié, avec une seule occurrence de ce caractère
tr-help-truncate-set1 = d'abord tronquer ENSEMBLE1 à la longueur d'ENSEMBLE2
tr-help-unbuffered = garantir que la sortie n'est pas mise en mémoire tampon

# Messages d'erreur
tr-error-missing-operand = opérande manquant
Expand Down
43 changes: 42 additions & 1 deletion src/uu/tr/src/tr.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ use operation::{
};
use simd::process_input;
use std::ffi::OsString;
#[cfg(not(any(unix, target_os = "wasi")))]
use std::io::Write;
use std::io::{stdin, stdout};
use uucore::display::Quotable;
use uucore::error::{UResult, USimpleError, UUsageError};
Expand All @@ -27,6 +29,7 @@ mod options {
pub const DELETE: &str = "delete";
pub const SQUEEZE: &str = "squeeze-repeats";
pub const TRUNCATE_SET1: &str = "truncate-set1";
pub const UNBUFFERED: &str = "unbuffered";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should drop long option since we need BSD compatibility only.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There isn't one, "unbuffered" is only the argument ID (like SETS = "sets"), there's no .long(). tr --unbuffered is rejected with unexpected argument '--unbuffered' found, only -u is accepted.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry.

pub const SETS: &str = "sets";
}

Expand Down Expand Up @@ -99,10 +102,15 @@ pub fn uumain(args: impl uucore::Args) -> UResult<()> {
let mut locked_stdin = stdin.lock();
// Write straight to the file descriptor: `Stdout` is line buffered, which
// costs a search for the last newline and an extra write per chunk.
// Every chunk is written as soon as it is translated, so `-u` has nothing
// left to do here.
#[cfg(any(unix, target_os = "wasi"))]
let mut output = uucore::io::RawWriter(stdout());
#[cfg(not(any(unix, target_os = "wasi")))]
let mut output = stdout().lock();
let mut output = FlushEachChunk {
inner: stdout().lock(),
enabled: matches.get_flag(options::UNBUFFERED),
};

// According to the man page: translating only happens if deleting or if a second set is given
let translating = !delete_flag && sets.len() > 1;
Expand Down Expand Up @@ -161,6 +169,33 @@ pub fn uumain(args: impl uucore::Args) -> UResult<()> {
Ok(())
}

/// `Stdout` holds back what follows the last newline; with `-u`, flush it
/// after every chunk so that a partial line is not held back either.
#[cfg(not(any(unix, target_os = "wasi")))]
struct FlushEachChunk<W: Write> {
inner: W,
enabled: bool,
}

#[cfg(not(any(unix, target_os = "wasi")))]
impl<W: Write> Write for FlushEachChunk<W> {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
self.inner.write(buf)
}

fn write_all(&mut self, buf: &[u8]) -> std::io::Result<()> {
self.inner.write_all(buf)?;
if self.enabled {
self.inner.flush()?;
}
Ok(())
}

fn flush(&mut self) -> std::io::Result<()> {
self.inner.flush()
}
}

pub fn uu_app() -> Command {
Command::new("tr")
.version(uucore::crate_version!())
Expand Down Expand Up @@ -203,6 +238,12 @@ pub fn uu_app() -> Command {
.action(ArgAction::SetTrue)
.overrides_with(options::TRUNCATE_SET1),
)
.arg(
Arg::new(options::UNBUFFERED)

@oech3 oech3 Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mask this from manpage by .hide(true). While silently supporting is useful for existing scripts, we should not encourage usage of it, or many people would write scripts incompatible with GNU tr.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, done in 742af96. -u is gone from --help and the man page but still accepted. I kept the short note in docs/src/extensions.md since Devel08 asked for it in the issue, happy to drop it if maintainers would rather leave it fully undocumented.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can drop it from extensions.md after GNU supported it.

.short('u')
.help(translate!("tr-help-unbuffered"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed if it is hidden?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for --help or the man page, but uudoc still lists hidden options in the zsh/fish/PowerShell completions and on the generated docs page, so -u shows up there either way. Without the text it would just be a bare -u with no explanation; with it, someone who runs into -u in a macOS script can see what it does. It also means unhiding it later is only removing .hide(true). more -u and uname -p/-i keep their help text while hidden too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SETS)
.num_args(1..)
Expand Down
47 changes: 47 additions & 0 deletions tests/by-util/test_tr.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1766,6 +1766,53 @@ fn test_failed_write_is_reported() {
.stderr_is("tr: write error: No space left on device\n");
}

#[test]
fn test_unbuffered_flag() {
new_ucmd!()
.args(&["-u", "a-z", "A-Z"])
.pipe_in("hello\n")
.succeeds()
.stdout_only("HELLO\n");
}

#[test]
#[cfg_attr(wasi_runner, ignore = "WASI: no pipe support")]
fn test_unbuffered_partial_line() {
// With -u, a chunk without a trailing newline must come out before more
// input arrives, as in `tail -f log | tr -u a-z A-Z | consumer`.
//
// Run in a separate thread so that the test fails via timeout rather
// than hanging if tr holds the output back.
let handle = std::thread::spawn(|| {
for (args, out1, out2) in [
(&["-u", "a-z", "A-Z"][..], "ABC", "DEF\nXY"),
(&["-u", "-d", "e"][..], "abc", "df\nxy"),
(&["-u", "-s", "a-z"][..], "abc", "def\nxy"),
] {
let mut child = new_ucmd!()
.args(args)
.set_stdin(std::process::Stdio::piped())
.set_stdout(std::process::Stdio::piped())
.run_no_wait();
child.write_in(b"abc");
assert_eq!(child.stdout_exact_bytes(out1.len()), out1.as_bytes());
child.write_in(b"def\nxy");
assert_eq!(child.stdout_exact_bytes(out2.len()), out2.as_bytes());
child.wait().unwrap().success();
}
});

for _ in 0..500 {
std::thread::sleep(std::time::Duration::from_millis(10));
if handle.is_finished() {
break;
}
}

assert!(handle.is_finished(), "tr -u held back a partial line");
handle.join().unwrap();
}

#[test]
#[cfg_attr(wasi_runner, ignore = "WASI: no pipe/signal support")]
fn test_broken_pipe_no_error() {
Expand Down
Loading