Let me show you what I mean by giving an example:

# Assume we have this list of increasing numbers
140
141
145
180
190
...

# If we pick 150 as threshold, the output should consist of 145 and all larger values:
145
180
190
...

# In the edge case where 150 itself is in the list, the output should start with 150:
150
180
190
...

I guess one can always hack something together in awk with one or two track-keeper variables and a bit of control logic. However, is there a nicer way to do this, by using some nifty combination of simpler filters or awk functionalities?

One way would be to search for the line number n of the first entry larger than the threshold, and then print all lines starting with n-1. What command would be best suited for that?

Still, I’m also wondering: Can awk or any other standard tool do something like “for deciding whether to print this line, look at the next line”?

(In my use case, the list is short, so performance is not an issue, but maybe let’s pretend that it were. Also, in my use case, all entries are unique, but feel free to work without this assumption.)

  • EpheraEnglish
    2•1 day ago

    Still, I’m also wondering: Can awk or any other standard tool do something like “for deciding whether to print this line, look at the next line”?

    I don’t know enough about awk to think this through to the end, but a trick from functional programming, which you might be able to apply here, is to ‘zip’ the list with itself, offset by one.

    So, it might then look like this, for example:

    140,141
141,145
145,180
180,190
190,

    Of course, in Bash you would probably need to implement the zipping yourself, so it might not save you much effort here…

    • @loveknight@programming.devOPEnglish
      3•8 hours ago

      Thank you, in fact I ended up doing something that’s mathematically pretty much just that: I have the previous line stored in an auxiliary variable lastline, and it is the evaluation of the current line $0 that determines whether the previous line gets printed.

      awk -v threshold=150 'BEGIN {lastline=""}
  (lastline!="" && threshold<$0){print lastline} #the additional check lastline!="" prevents an empty line at the very beginning
  {lastline=$0}
  END{print} #hardcode printing of the very last line, because otherwise it would never be printed
'

      Of note, in the case where some list entries are repeated, the behavior of this script will be:

      • The threshold value, if it’s in the list, will always be printed just once, even if it occurs multiple times in the list, and also if it happens to be the first, last, or only entry in the list.
      • All larger entries will be printed exactly as often as they occur in the list. This even holds for the largest value: its last repetition will be printed via the final END{print} statement, whereas all preceding instances get printed through the statement that depends on threshold<$0.

      (IIRC, it was a StackOverflow post that led me to this.)