text processing utilities

Highlight documentation

Highlight News

Some details of the recent changes mentioned in the ChangeLog.

Service Mode

Since version 4.6 highlight can be invoked as long running process to process files using stdin and stdout. This might result in better performance, ie. when used in a file viewer application.

Service mode (`--service-mode`) is designed to allow several inputs to be parsed on demand from an external application without restarting highlight each time. Service mode runs continuously until a kill signal is sent, EOF is read from STDIN, or the "exit" command is sent instead of a mode line specified below. Service mode expects each input to be separated by a EOF character on a line by itself (if EOF was set to '\0' then "\n\0\n"). Before each file (including the first file) a new mode line must be sent with any mode changes to occur (even if no changes are desired a new line must be sent).

Mode options are changed with key value pairs like `option=value` separated by a semicolon (no trailing semicolon for last item needed). Any options not changed will retain their previous value.

The following options are supported: 

`syntax=file.ext` ::: to change the syntax to the extension from the filename,
or syntax= to set the syntax to use directly (ie csharp).

`tag=##` ::: if provided and not empty will be printed by itself on a new line
to stdout after a file has been processed.

`eof=##` ::: A single character (0x00-0xFE) that stdin will contain to separate
one file from the next. Recommend 0x00 if you can send it, otherwise 0x01-0x07
are generally easy for most clients to send, ie 0x07 ('\a') for bell. It is
near impossible to send several characters above 128 in windows through most
term emulators.

Note the EOF char must occur on its own new line it cannot be found mid-line. There is no need for a new line between the previous EOF and the next mode options line. You also cannot send the EOF char to new line as it is used as a delimiter already as described.

Python sample script

#!/usr/bin/env python
import sys
from subprocess import Popen, PIPE

# Script to show the new service mode based on stdin/stdout
# requires highlight 4.6

# Add for custom paths: '-D', '/usr/share/highlight/'
p = Popen( ["highlight",'--service-mode', '--out-format', 'xterm256', '-f'],
		  stdin=PIPE, stdout=PIPE, stderr=PIPE, bufsize=1, universal_newlines=True )

endtag = '<endtag/>'

while True:
    filename = input("\nPlease enter the filename (or exit): ")

    if filename=='exit':
        break

    p.stdin.write(str('eof=\x01;tag='+endtag+';syntax='+filename+'\n'))
    p.stdin.flush()

    # write file
    with open(filename) as file:
        while line := file.readline():
            p.stdin.write(line)

    # send end of file delimiter and flush
    p.stdin.write(str('\n\x01\n'))
    p.stdin.flush()

    # read highlight output
    while True:
        result = p.stdout.readline().strip()

        if result==endtag:
            break

        print(result)


for pipe in [p.stdin, p.stdout]:
    try:
        pipe.close()
    except OSError:
        pass

sys.exit(p.wait())
Tupel7