     [file version.texi not included]

This manual is for ctlseqs, a helper library for control sequences.

Copyright © 2021,2022  CismonX <admin@cismon.net>

     Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.  A copy of the license is included in the section entitled ``GNU Free Documentation License''.

ctlseqs
*******

This manual is for ctlseqs, a helper library for control sequences.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.

ctlseqs is free software. You can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Overview of ctlseqs
*******************

The name ``ctlseqs'' is an abbreviation of ``control sequences'', as defined in section 5.4 of ECMA-48.

As the name suggests, this library focuses on handling control sequences. However, it only cares about the bit combinations, while the actual meaning and implementation of a control sequence is up to the user.

The C API provided by ctlseqs is composed of three major parts: The helper macros, the control sequence matcher, and the control sequence reader. Any of them can be used separatedly or combined, after including the header file ctlseqs.h in a source file.

Contributing
============

We welcome any form of contribution to ctlseqs (as well as this manual), including bug reports, patches, etc.

Source code of ctlseqs is hosted on Savannah (https://savannah.nongnu.org/projects/ctlseqs). You can contribute to ctlseqs using the bug tracker and patch manager, or discuss with the community using the mailing lists.

Use Scenarios of ctlseqs
========================

Control sequences, as well as other control functions, were once commonly used in computer terminals. Terminals exchange control information with the host regarding colors, font styles, cursor position, etc., using control functions embedded in normal text. Such physical terminals are no longer used today, however, popular ones like DEC VT100 are widely emulated by modern terminal emulators.

The primary purpose of the ctlseqs library is to provide developers with simple and easy-to-use API for handling control functions, when working on terminal emulators and text-based programs.

However, since there is no de facto standard, control functions used in terminals are largely vendor-specific, and terminal emulators like to add their own private controls. That makes ctlseqs not suitable for writing text-based programs which intend to be portable. Instead of raw control codes, developers should stick to ncurses or terminfo.

There are still cases when dealing with raw escape sequences is inevitable, and ctlseqs may come in handy:

  * Development of text-based programs which rely heavily on special control sequences, which is not supported by libraries like ncurses.

  * Implementing a terminal emulator.

  * Experimenting or debugging the features of text-based programs or terminal emulators.

Helper Macros
*************

ctlseqs provides C preprocessor macros representing control functions, which expand to C string literals.

The control function can be one of the following three types:

  * Elements from the C0 or C1 set.

  * Control Sequences.

  * Other control functions (such as device control functions).

The name of a helper macro is the control function name with CTLSEQS_ as prefix. For a control function other than elements from the C0 or C1 set, the corresponding helper macro is a function-like macro which may or may not take arguments.

Control sequences listed in the helper macros are primarily excerpted from XTerm's manual (https://invisible-island.net/xterm/ctlseqs/ctlseqs.html), which may differ across implementations.

As ctlseqs does not currently support 8-bit controls, 2-character 7-bit codes from the C1 set are used instead of their 1-character 8-bit representation. For example, CTLSEQS_CSI expands to "\x1b[".

The following code snippet is an example usage of helper macros:

     printf(CTLSEQS_BEL);
     printf(CTLSEQS_XTVERSION());
     printf(CTLSEQS_CUP("%d", "%d"), 3, 4);

Keep in mind that the standard output stream is by default line buffered in a terminal. In order to make the control functions take effect immediately, either fflush(stdout) after printing, or disable output buffering with setvbuf(stdout, NULL, _IONBF, 0).

Control Sequence Matching
*************************

Given a character string, checking whether it matches a control sequence is quite trivial, with only the standard C library:

     char const *str /* = ... */;
     int row, col;
     if (0 == strcmp(str, CTLSEQS_XTVERSION())) {
         // ...
     } else if (2 == sscanf(str, CTLSEQS_CUP("%d", "%d"), &row, &col)) {
         // ...
     } else {
         // ...
     }

However, as the number of patterns grows, this naive implementation becomes less efficient and harder to maintain. This problem can be easily solved by using the control sequence matcher provided by ctlseqs.

The struct ctlseqs_matcher * is a pointer to an opaque type which represents an instance of control sequence matcher. Before using, the matcher should be initialized with ctlseqs_matcher_init. After using, it should be deallocated with ctlseqs_matcher_free.

     struct ctlseqs_matcher *matcher = ctlseqs_matcher_init();
     // ...
     ctlseqs_matcher_free(matcher);

On rare occurences when ctlseqs fail to allocate enough memory, function ctlseqs_matcher_init may return NULL. However, it is safe to pass null pointers to ctlseqs_matcher_free.

Matcher Configuration
=====================

Matcher configuration consists of two parts: the number of matching patterns, and the pattern values. Invoke function ctlseqs_matcher_config to configure a matcher.

     struct ctlseqs_matcher *matcher /* = ... */;
     char const *patterns[] = {
         // ...
     };
     struct ctlseqs_matcher_options options = {
         .patterns  = patterns,
         .npatterns = sizeof(patterns) / sizeof(char const *),
     };
     int result = ctlseqs_matcher_config(matcher, &options);
     // ...

Each invocation of ctlseqs_matcher_config on the same matcher overwrites the data generated from the last invocation. Upon success, the function returns CTLSEQS_OK. If the function fails to allocate enough memory, returns CTLSEQS_NOMEM.

     If the patterns field in struct ctlseqs_matcher_options is invalid, function behaviour is undefined. See Patterns for details.

Patterns
--------

The patterns field in struct ctlseqs_matcher_options is an array of NUL-terminated strings which indicates the desired patterns of control functions for the current matcher.

The following types of control functions are recognizable by the matcher:

  * Control sequences: CSI [param...] [intmd...] final

  * C1 functions with command string: (APC|DCS|OSC|PM) [cmdstr] ST

  * Single shifts: (SS2|SS3) ch

  * SOS function: SOS [chrstr] ST

According to ECMA-48, CSI parameter bytes are of range 0x30 to 0x3f, intermediate bytes 0x20 to 0x2f, and final byte 0x40 to 0x7e. Command string consists of printable characters and characters of range 0x08 and 0x0e. Character string can be any bit combination which does not represent SOS or ST.

A supported control function, either verbatim or combined with placeholders, can be specified as a valid pattern. The terminating NUL character does not count into the pattern.

A placeholder indicates that when matching a string against the pattern, the value at the placeholder's location should conform to its rules. A placeholder can only take place in the param, intmd, cmdstr or chrstr fields, and can be one of the following values:

  * CTLSEQS_PH_NUM: An unsigned integer.

  * CTLSEQS_PH_NUMS: Multiple unsigned integers separated with the semicolon ASCII character (value 0x3b).

  * CTLSEQS_PH_STR: A string of printable characters.

  * CTLSEQS_PH_CMDSTR: A string containing only printable characters and characters of range 0x08 to 0x0d.

  * CTLSEQS_PH_CSI_PARAM: A string of CSI parameter bytes.

  * CTLSEQS_PH_CSI_INTMD: A string of CSI intermediate bytes.

  * CTLSEQS_PH_HEXNUM: A string representing a hexadecimal number.

  * CTLSEQS_PH_CHRSTR: A string of any bit combination which does not represent SOS or ST.

The following code is a valid example of patterns:

     const char *patterns[] = {
         CTLSEQS_CUP(CTLSEQS_PH_NUM, CTLSEQS_PH_NUM),
         CTLSEQS_XTVERSION(),
         CTLSEQS_DECRQM("1000"),
         // ...
     };

Matching String
===============

Function ctlseqs_match matches a given character string to a matcher. The function accepts four arguments: the matcher, the string to match, length of the string to match, and a buffer which stores the match result.

Before matching, a buffer which is large enough to store the match result should be allocated. The buffer is an array of union ctlseqs_value, whose definition is shown below:

     union ctlseqs_value {
         char const    *str;
         size_t         len;
         unsigned long  num;
     };

If the string contains a recognizable control function, or part of a control function which is not yet terminated by the end of the string, the length of control function will be stored at len field of match result buffer at offset 0, and the pointer to the first character of the control funtion at the str field at offset 1.

If ctlseqs_match fails to find any control functions, returns CTLSEQS_NOSEQ. For a partial control function, returns CTLSEQS_PARTIAL. If the matcher is not configured with a matching pattern of the control function, the function returns CTLSEQS_NOMATCH.

If the control function matches a pattern configured in the matcher, returns the offset of the matched pattern, and stores the extracted values to the result buffer according to each of the placeholders, starting from offset 2:

  * CTLSEQS_PH_NUM, CTLSEQS_PH_HEXNUM: The integer is stored in field num.

  * CTLSEQS_PH_NUMS: The number of integers is stored in field len, followed by that many integers stored in field num.

  * CTLSEQS_PH_CSI_PARAM, CTLSEQS_PH_CSI_INTMD, CTLSEQS_PH_STR, CTLSEQS_PH_CMDSTR, CTLSEQS_PH_CHRSTR: The length of the string is stored in field len, followed by a pointer to the first character of the string stored in field str.

The following code is an example of invoking ctlseqs_match:

     union ctlseqs_value     buffer[4];
     struct ctlseqs_matcher *matcher /* = ... */;
     // ...
     char const *str     = "foo" CTLSEQS_CUP("2", "4");
     size_t      str_len = sizeof("foo" CTLSEQS_CUP("2", "4")) - 1;
     ssize_t     result  = ctlseqs_match(matcher, str, str_len, buffer);

     assert(result == 0);
     assert(buffer[0].len == 6 && buffer[1].str == str + 3);
     assert(buffer[2].num == 2 && buffer[3].num == 4);

Function ctlseqs_match allows NULL value for argument matcher, in which case it behaves like a matcher configured with zero patterns is provided.

     If the given string can match multiple patterns in the matcher, it is unspecified which one of them will be the final match result.

Control Sequence Reading
************************

In practice, control sequences are often read from stream (e.g. STDIN). The ctlseqs library provides a control sequence reader utility, which reads data from a file descriptor, buffers it, and matches the data against the patterns specified in the given matcher.

Like the matcher, struct ctlseqs_reader * is a pointer to an opaque type which represents an instance of control sequence reader, initialized with ctlseqs_reader_init and deallocated with ctlseqs_reader_free after using. It is safe to pass null pointers to ctlseqs_reader_free.

     struct ctlseqs_reader *reader = ctlseqs_reader_init();
     // ...
     ctlseqs_reader_free(reader);

Reader Configuration
====================

Options of a control sequence reader can be set with function ctlseqs_reader_config.

     union ctlseqs_value buffer[4];

     struct ctlseqs_reader *reader /* = ... */;
     struct ctlseqs_reader_options options = {
         .result = buffer,
         .maxlen = 1024,
         .fd     = STDIN_FILENO,
         .flags  = 0,
     };
     int result = ctlseqs_reader_config(reader, &options);
     // ...

In struct ctlseqs_reader_options:

  * field result is the buffer where match results are stored, as defined in Matching String.

  * Field maxlen is the maximum length of control sequence to process, before the reader gives up reading and returns an error.

  * Field fd is the file descriptor to read from.

  * Field flags is the bitmask of multiple boolean options that affects the reader's behaviour. See Reader Flags for details.

Upon success, the function return CTLSEQS_OK. If the function fails to allocate enough memory (for the internal read buffer), returns CTLSEQS_NOMEM. If trying to set maxlen to a value smaller than the current internal read buffer data size, returns CTLSEQS_ERROR.

Reader Flags
------------

Tips & Hints
************

Example Programs
****************

In the source code repository of ctlseqs, there are a few programs that demonstrates the basic usage of ctlseqs.

See ``example/sixdraw.c'' for a simple TUI program in which you can draw lines on the terminal window using a mouse.

See ``tests/tcsgrep.c'' for a program which parses control functions into human-readable text. It was originally written as a tool to run tests for ctlseqs, but can also be considered as a crude alternative of GNU Teseq (https://www.gnu.org/software/teseq/).

API Reference
*************

This section contains a complete list of functions exposed by ctlseqs.  See the corresponding man pages for details.

Initialize matcher (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_matcher_init.3.html):

     struct ctlseqs_matcher *ctlseqs_matcher_init(void);

Configure matcher (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_matcher_config.3.html):

     int ctlseqs_matcher_config(
         struct ctlseqs_matcher               *matcher,
         struct ctlseqs_matcher_options const *options
     );

Match string (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_match.3.html):

     ssize_t ctlseqs_match(
         struct ctlseqs_reader const *matcher,
         char                  const *str,
         size_t                       str_len,
         union ctlseqs_value         *result
     );

Destroy matcher (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_matcher_free.3.html):

     void ctlseqs_matcher_free(
         struct ctlseqs_matcher *matcher
     );

Initialize reader (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_reader_init.3.html):

     struct ctlseqs_reader *ctlseqs_reader_init(void);

Configure reader (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_reader_config.3.html):

     int ctlseqs_reader_config(
         struct ctlseqs_reader               *reader,
         struct ctlseqs_reader_options const *options
     );

Read and match (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_read.3.html):

     ssize_t ctlseqs_read(
         struct ctlseqs_reader        *reader,
         struct ctlseqs_matcher const *matcher,
         int                           timeout
     );

Purge reader (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_purge.3.html):

     void ctlseqs_purge(
         struct ctlseqs_reader *reader,
         size_t                 nbytes
     );

Destroy reader (https://nongnu.org/ctlseqs/man-pages/man3/ctlseqs_reader_free.3.html):

     void ctlseqs_reader_free(
         struct ctlseqs_reader *reader
     );

GNU Free Documentation License
******************************

     [file fdl.texi not included]