@c AnsiCmd Library User's Guide  - - - - - - - - - - - - - - - - - - - - - - - -
@c 08.0 CsvView Application  - - - - - - - - - - - - - - - - - - - - - - - - - -
@c      View CSV files (AnsiCmd basic functionality) - - - - - - - - - - - - - -
@node CsvView Demo App
@unnumbered CsvView Demo App
@cindex 08.0.00 CsvView Demo App

@b{@t{'CsvView' (csvv)}} reads any Comma-Seperated-Values file and formats the 
data for viewing in the terminal window.

@menu
* Introduction to CsvView::   A description of basic functionality
* Invoking CsvView::          Command line options
* CSV Encoding::              Examples of CSV-encoding formats
* Build CsvView from Source:: Building CsvView from Source
@end menu
@sp 2

@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@lowersections

@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@c Chapter 08.01 Introduction to CsvView - - - - - - - - - - - - - - - - - - - -
@c               A description of basic functionality  - - - - - - - - - - - - -
@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@node Introduction to CsvView
@unnumbered Introduction to CsvView
@cindex 08.01 Introduction to CsvView

@Subheading{What Is CSV?,------------}

Comma-Seperated-Values files are plain-text files commonly used for transferring 
tabular data between applications or accross the web. The format is very simple 
and supported by many applications such as LibreOffice, Excel(tm), as well as 
various email platforms and database applications.

While there is no formal specification for CSV data, several groups have worked 
toward a normalized format, most notably the World Wide Web Consortium (W3C).@*
@ifnothtml
@verb{    } @url{https://www.w3.org/TR/tabular-data-primer/}@*
@verb{    } @url{https://www.w3.org/2013/csvw/wiki/Main_Page.html}
@end ifnothtml
@ifhtml
@html
<span style="margin-left:2.5em;"><a href="https://www.w3.org/TR/tabular-data-primer/" 
target="_blank">https://www.w3.org/TR/tabular-data-primer/</a></span><br>
<span style="margin-left:2.5em;"><a href="https://www.w3.org/2013/csvw/wiki/Main_Page.html" 
target="_blank">https://www.w3.org/2013/csvw/wiki/Main_Page.html/</a></span>
@end html
@end ifhtml

Additional information (and the opinions of various parties) are available:@*
Internet Engineering Administration, Research Task Force @t{(IRTF)}:@*
@ifnothtml
@verb{    } @url{https://datatracker.ietf.org/doc/html/rfc4180}@*
@end ifnothtml
@ifhtml
@html
<span style="margin-left:2.5em;"><a href="https://datatracker.ietf.org/doc/html/rfc4180" 
target="_blank">https://datatracker.ietf.org/doc/html/rfc4180</a></span><br>
@end html
@end ifhtml
FileFormat.com:@*
@ifnothtml
@verb{    } @url{https://docs.fileformat.com/spreadsheet/csv/}@*
@end ifnothtml
@ifhtml
@html
<span style="margin-left:2.5em;"><a href="https://docs.fileformat.com/spreadsheet/csv/" 
target="_blank">https://docs.fileformat.com/spreadsheet/csv/</a></span><br>
@end html
@end ifhtml
Wikipedia:@*
@ifnothtml
@verb{    } @url{https://en.wikipedia.org/wiki/Comma-separated_values}
@end ifnothtml
@ifhtml
@html
<span style="margin-left:2.5em;"><a href="https://en.wikipedia.org/wiki/Comma-separated_values" 
target="_blank">https://en.wikipedia.org/wiki/Comma-separated_values</a></span><br>
@end html
@end ifhtml

For examples of CSV files, please @pxref{CSV Encoding}, and the sample 
CSV files included with this distribution.

@Subheading{The CsvView Application,-----------------------}
@cindex output formats

The @t{CsvView} application (csvv) reads any CSV-formatted file and reformats it 
for viewing, either directly in the terminal window or through the @t{'less'} 
utility. @t{CsvView} supports source files in any of the four(4) authorized 
encodings:@*
@b{@verb{|   UTF-8|}}@*
@b{@verb{|   UTF-8 with byte-order mark (BOM)|}}@*
@b{@verb{|   UTF-16LE (little-endian)|}}@*
@b{@verb{|   UTF-16BE (big-endian)|}}

@cindex less utility
It is important to note that the @t{'less'} can display @emph{some} CSV files 
without the need for reformatting; however, @t{'less'} will often complain 
that the target file is a @t{"binary"} file, and cannot display the data as text, 
even though the file does not actually contain binary data.
@anchor{Output Formats}

For this reason, @t{CsvView} reformats the source data into pure UTF-8 encoding 
before display. This re-encoding also vertically aligns the fields to present 
the data in an orderly manner. @t{CsvView} can format the output in any of 
three(3) ways.

@b{Horizontal Format: one record per line}@*
@verbatim
NAME              TITLE                PHONE        EMAIL
Helena Handbasket Queen of the Castro  415-654-3210 Yaz-Queen@TwinPeaks.com
@end verbatim

@b{Vertical Format : one field per line}@*
@verbatim
NAME     Helena Handbasket 
TITLE    Queen of the Castro
PHONE    415-654-3210
EMAIL    Yaz-Queen@TwinPeaks.com
@end verbatim

@b{Quick Format : horizontal format, truncated to width of window}@*
@verbatim
┌───────────────────────────────────────────────────────┐
│NAME              TITLE                PHONE        EMA│
│Helena Handbasket Queen of the Castro  415-654-3210 Yaz│
└───────────────────────────────────────────────────────┘
@end verbatim

Please @pxref{csv display option} for selection of the display format.

@t{CsvView} may also be used to @b{@emph{create}} files in CSV format using 
any of the encoding methods described above.@*
Please @pxref{csv create option} for details.
@sp 2


@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@c Chapter 08.02 Invoking CsvView  - - - - - - - - - - - - - - - - - - - - - - -
@c               Command line options  - - - - - - - - - - - - - - - - - - - - -
@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@node Invoking CsvView
@unnumbered Invoking CsvView
@cindex 08.02 Invoking CsvView
@cindex invoking CsvView
@cindex CsvView invoking

This chapter lists the command-line options for invoking the @t{CsvView} (csvv).
The @t{'help'} and @t{'version'} options override all other specified options. 

The @t{CsvView} application is intentionally very simple @EMDASH built without 
support for direct user-configuration of the @t{AnsiCmd} library. The only 
available terminal configuration option is for selection of the color 
attributes used for text output.@*
Please @pxref{Invoking - Terminal Setup} for a detailed discussion of the 
@t{AnsiCmd} terminal setup options used for the @t{EarthPoints} and @t{AcEdit} 
applications,@*


@Subheading{Summary of Options,------------------}

@indentedblock
@multitable {xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} {xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}
@headitem @ @ OPTION @tab @ @ DESCRIPTION
@item @ @ @ref{csv filespec option}       @tab @ Specify the source CSV file
@item @ @ @ref{csv display option}        @tab @ Specify the display format
@item @ @ @ref{csv fgndattr option}       @tab @ Foreground color attribute
@item @ @ @ref{csv bgndattr option}       @tab @ Background color attribute
                                               @ to a file
@item @ @ @ref{csv language option}       @tab @ Specify user-interface
                                               @ language
@item @ @ @ref{csv header option}         @tab @ Specify whether source file
                                               @ includes a header record
@item @ @ @ref{csv saveas option}         @tab @ Save formatted display data
@item @ @ @ref{csv stats option}          @tab @ Report source file stats
@item @ @ @ref{csv dimensions option}     @tab @ Report dimensions of terminal
                                               @ window
@item @ @ @ref{csv help option}           @tab @ Display all application
                                               @ options
@item @ @ @ref{csv helpless option}       @tab @ Same as @t{'--help'} option
                                               @ except pipe through @t{'less'}
@item @ @ @ref{csv version option}        @tab @ Reports CsvView version 
                                               @ number and copyright info.
@item @ @ @t{---} Development Options @t{---}   @tab @ @verb{|    ------------------|}
@item @ @ @ref{csv create option}         @tab @ Create a CSV file
@item @ @ @ref{csv capture option}        @tab @ Capture a range of records
@item @ @ @ref{csv terminfo option}       @tab @ Report the current
                                               @ terminal configuration
@item @ @ @ref{csv pause option}          @tab @ Pause before display of
                                               @ formatted data
@end multitable
@end indentedblock

Options may be specified in any order.

For each option that requires one or more arguments, the option name and 
its argument(s) may be separated by either @t{'='} or @t{' '} (space). 
@ifnothtml
@*
@end ifnothtml
Note that if the argument contains whitespace, then the entire argument 
sequence must be enclosed within single-quotes or double-quotes. 
@ifnothtml
@*
@end ifnothtml
This prevents the terminal's shell program from @i{@LDQ{}interpreting@RDQ{}} 
(trashing) your argument formatting.

Multiple instances of an option will result in unpredictable results, 
so don't do it.

@verbatim
Examples:
    --filespec=Contacts.csv
    --filespec Contacts.csv
    --create=Sample.txt,utf8bom,crlf
    --create "Sample Records.txt, utf16le, crlf"
@end verbatim

@b{Note:} As a convenience, the options may be uniquely specified 
in a truncated form. At least six (6) characters of the option must be 
specified. This consists of the two @t{'dash'} characters, plus the first four 
characters of the option name. Example: The @LDQ{}@b{@t{--filespec}}@RDQ{} 
option may be uniquely specified as @LDQ{}@b{@t{--file}}@RDQ{}.
@sp 2

@Subheading{Invocation Options,------------------}
@sp 1

@c = = = = = = = = = = = = = =
@c =  --filespec option      =
@c = = = = = = = = = = = = = =
@cindex csv filespec option
@cindex option, csv filespec
@anchor{csv filespec option}
@ifnothtml
@w{ @minus{}@minus{}filespec  Specify the source CSV file.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;filespec</span>
<span style="margin-left:2.0em;">
Specify the source CSV file.
</span></span>
@end html
@indentedblock
Specify the file to be displayed.@*
The source file may be specified as a filename in the current directory, 
as a relative path, or as an absolute path. Special characters such as 
@t{'.' '..' '~'} are recognized, and symbolic links in the path will be followed. 
The filename must include the @t{".csv"} filename extension.

The file must be in Comma-Seperated-Values (CSV) format, and encoded as@*
@t{UTF-8, UTF-8 with byte-order-mark, UTF-16LE or UTF-16BE}.@*
If the file cannot be verified as containing valid CSV data, a warning will 
be displayed and the application will exit.@*
@b{Important Note:} Because there is no formal standard for the CSV format, it 
is possible that a non-CSV file will be accepted for processing. If this 
happens, then as the saying goes: @LDQ{}Garbage in, garbage out.@RDQ{}

@verbatim
   EXAMPLES:  csvv --filespec=Contacts.csv
              csvv --filespec='Company Employee Info.csv'
              csvv --filespec "Fred's Extended Family.csv"
              csvv --file=Contacts.csv
   Although it is not recommended, CsvView will accept a source
   filename WITHOUT the "--filespec" option:
              csvv Contacts.csv --display=v --stats
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =   --display option      =
@c = = = = = = = = = = = = = =
@cindex csv display option
@cindex option, csv display
@anchor{csv display option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}display  Specify the display format.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;display</span>
<span style="margin-left:2.0em;">
Specify the display format.
</span></span>
@end html
@indentedblock
The @t{display} option specifies the type of formatting to be used for display 
of the CSV records. The options are:@*
@format
@ @b{h} @EMDASH Horizontal (default)
@t{    } Display one record per line.
@t{    } The record fields are aligned vertically for ease of viewing.
@t{    } The data are displayed using the @t{'less'} utility to fascilitate 
@t{    } both vertical and horizontal scrolling.
@ @b{v} @EMDASH Vertical
@t{    } Display one field per line. This is useful if the record
@t{    } is wider than the width of the terminal window.
@t{    } The data are displayed using the @t{'less'} utility to fascilitate 
@t{    } both vertical and horizontal scrolling.
@ @b{q} @EMDASH Quick
@t{    } The Quick output format is the same as the Horizontal format except:
@t{    } a) The data are written directly into the terminal window, 
@t{       } one page at a time.
@t{       } This is similar to viewing data using the @t{'cat'} utility.
@t{    } b) The data are truncated at the right edge of the terminal window
@t{       } to prevent the terminal from inserting automatic line breaks.
@end format

Please @pxref{Output Formats} for further information.

@verb{|   EXAMPLE: csvv --display=v|}
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =   --fgndattr option     =
@c = = = = = = = = = = = = = =
@cindex csv fgndattr option
@cindex option, csv fgndattr
@anchor{csv fgndattr option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}fgndattr  Set foreground color attribute.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;fgndattr</span>
<span style="margin-left:2.0em;">
Set foreground color attribute.
</span></span>
@end html
@indentedblock
Specify the foregroud (text) attribute used for display of the data.@*
By default, the text data are displayed using @b{@emph{blue}} text.

The available attributes are the eight(8) basic colors and their 
@LDQ{}intense@RDQ{} (bold) variants, or the terminal-default attribute.

@verbatim
   BASIC    INTENSE
   -----    -------
   black    blacki
   red      redi
   green    greeni
   brown    browni
   blue     bluei
   magenta  magentai
   cyan     cyani
   grey     greyi
   dflt

   EXAMPLES:  csvv --fgndattr=green
              csvv --fgndattr red
              csvv --fgndattr=redi
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =   --bgndattr option     =
@c = = = = = = = = = = = = = =
@cindex csv bgndattr option
@cindex option, csv bgndattr
@anchor{csv bgndattr option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}bgndattr  Set background color attribute.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;bgndattr</span>
<span style="margin-left:2.0em;">
Set background color attribute.
</span></span>
@end html
@indentedblock
Specify the backgroud attribute used for display of the data.@*
By default, the background attribyte is the terminal default (@b{@emph{dflt}}).

The available attributes are the same as those listed for the foreground 
attributes above.@*
@b{Note:} Avoid using the same attribute for both foreground and background.
 
@b{[** THE bgndattr OPTION IS CURRENTLY IGNORED **]}
@verbatim
   EXAMPLES:  csvv --bgndattr=magenta
              csvv --bgndattr brown
              csvv --bgndattr=bluei
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =   --language option     =
@c = = = = = = = = = = = = = =
@cindex csv language option
@cindex option, csv language
@anchor{csv language option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}language  Specify the user interface language.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;language</span>
<span style="margin-left:2.0em;">
Specify the user interface language.
</span></span>
@end html
@indentedblock
Specify the language used by the application's user interface.

Options:@*
@verb{|   |}@t{@b{EN}   @EMDASH } ENglish@*
@verb{|   |}@t{@b{ZH}   @EMDASH } ZHongwen (中文) Simplified Chinese@*
@verb{|   |}@t{@b{tenv} @EMDASH } Terminal EnVironent (default)

By default, the user interface language used is derived from the locale used 
by the system. If the system locale is one of the Zhōngwén variants, the 
Chinese interface will be used; otherwise, the English interface will be used.@*
@xref{Terminal Configuration} for a discussion of locale.

To override the automatic language selection, use the @t{'--language'} option to 
select the desired user-interface language.

Note that the 'EN' and 'ZH' designators are the internationally accepted 
language designators.@*
@ifnothtml
Please refer to Library Of Congress document:@*
@url{https://www.loc.gov/standards/iso639-2/},@*
@end ifnothtml
@ifhtml
Please refer to Library Of Congress document:
@html
<span style="margin-left:1.0em;"><a href="https://www.loc.gov/standards/iso639-2/" 
target="_blank">https://www.loc.gov/standards/iso639-2/</a></span><br>
@end html
@end ifhtml
or to the Wikipedia article which describes the @t{ISO 639} standard:@*
@ifnothtml
@url{https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes}
@end ifnothtml
@ifhtml
@html
<span><a href="https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes" 
target="_blank">https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes</a></span><br>
@end html
@end ifhtml

@verbatim
   EXAMPLES:  csvv --language=ZH
              csvv --lang en
              csvv --language tenv
@end verbatim

@quotation Note
This application is designed to be as simple as possible, so we have implemented 
only our two(2) best languages. Other UI languages may be added in a later 
release.
@end quotation
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =    --header option      =
@c = = = = = = = = = = = = = =
@cindex csv header option
@cindex option, csv header
@anchor{csv header option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}header  Specify whether source contains header record.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;header</span>
<span style="margin-left:2.0em;">
Specify whether source contains header record.
</span></span>
@end html
@indentedblock
Override the application's determination of whether the source file contains 
a header record.@*
A header record contains column headings for the remainder of the records.

During the initial scan of the source file, the application tries to determine 
whether the first record of the file is a header record. The criteria for this 
test are:@*
@verb{|   1)|} Is the first line of the file terminated by a newline?@*
@verb{|   2)|} Does the record include at least one comma (',') delimeter?@*
@verb{|   3)|} Do all fields of the record contain text data, (non-empty)?

Clearly, the first record of a CSV file may meet all these criteria and still 
not be a header record@EMDASH{}or may be a header record without meeting these 
criteria. If this happens, the program's scan results may be overridden using 
the @t{'--header'} option to specify whether the file does, in fact include 
a header record.

Note that if a CSV file does not contain a header record, then horizontal 
output formatting will simply have no column headings; however, for vertical 
output formatting, generic column headings will be used. Example:@*
@verb{|   Field # 1  : Helena Handbasket|}@* 
@verb{|   Field # 2  : Queen of the Castro|}@* 
@verb{|   Field # 3  : 415-654-3210|}@* 
@verb{|   Field # 4  : Yaz-Queen@TwinPeaks.com|} 

@verb{|   EXAMPLE:  csvv --filespec='Contacts.csv' --header=y|}
@sp 1
@center @t{────────────────────}
@quotation
Note that the MIME-type database for the system may optionally specify whether 
CSV files on the system include header records; however, this information is 
not just optional, it is unreliable and is not referenced by this application. 
In a corporate setting, CSV files that are routinely used to transfer company 
data may adhere to the specified MIME type; however, a CSV file created by a 
non-corporate application (e.g. an email application), or a file randomly 
downloaded from the web are not likely to conform to the system's MIME-type database.
@end quotation
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =    --saveas option      =
@c = = = = = = = = = = = = = =
@cindex csv saveas option
@cindex option, csv saveas
@anchor{csv saveas option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}saveas  Save reformatted display data.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;saveas</span>
<span style="margin-left:2.0em;">
Save reformatted display data.
</span></span>
@end html
@indentedblock
When specified, the @t{'--saveas'} option saves a copy of the displayed data 
to a file in the current working directory. Specify the filename only. 

The saved data will be plain text in the same format used to display the data 
in the terminal window.@*
@c Note that the saved data @emph{are not} in CSV format.

The data format is determined by the @t{'--display'} option described above, 
either @t{'@b{Horizonal}'} or @t{'@b{Vertical}'}.@*
(For the @LDQ{}Quick@RDQ{} output format, the data are not saved.)

@verb{|   EXAMPLE:  csvv --filespec='contacts.csv' --saveas='formatted_data.txt'|}
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =     --help option       =
@c = = = = = = = = = = = = = =
@cindex csv help option
@cindex option, csv help
@anchor{csv help option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}help  (or @minus{}h)  Display command-line help.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;help &nbsp;&nbsp; (or &ndash;h)</span>
<span style="margin-left:2.0em;">
Display command-line help.
</span></span>
@end html
@indentedblock
Command-line Help. Display a brief summary of command-line usage and 
options.@*
(overrides everything on command line except @w{@t{'--version'})}

@verbatim
   EXAMPLES:  csvv --help
              csvv -h
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =   --helpless option     =
@c = = = = = = = = = = = = = =
@cindex csv helpless option
@cindex option, csv helpless
@anchor{csv helpless option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}helpless  (or @minus{}hl)  Display command-line help.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;helpless</span>
<span style="margin-left:2.0em;">
Display command-line help.
</span></span>
@end html
@indentedblock
Display command-line help. Same as the @t{'--help'} option above except 
that the help menu will be piped through the @t{'less'} utility.

@verbatim
   EXAMPLES:  csvv --helpless
              csvv -hl
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =    --version option     =
@c = = = = = = = = = = = = = =
@cindex csv version option
@cindex option, csv version
@anchor{csv version option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}version  Report CsvView version number.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;version</span>
<span style="margin-left:2.0em;">
Report CsvView version number.
</span></span>
@end html
@indentedblock
Display the CsvView version number and copyright information.@*
(overrides everything else on the command line)@*
The AnsiCmd-class and gString-class version numbers are also reported.

@verb{|   EXAMPLE:  csvv --version|}
@end indentedblock
@sp 1

@hr40{}

@c = = = = = = = = = = = = = =
@c =    --create option      =
@c = = = = = = = = = = = = = =
@cindex csv create option
@cindex option, csv create
@anchor{csv create option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}create  Create a CSV file from plain text.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;create</span>
<span style="margin-left:2.0em;">
Create a CSV file from plain text.
</span></span>
@end html
@indentedblock
Create a Comma-Seperated-Values (CSV) file using a plain-text file as the 
source data.

@ifnothtml
@example
┌───────────────────────────────────────────────────────┐
│ During development, this option was used to create    │
│ various test files to exercise the functionality      │
│ and general syntax checking of the application.       │
│                                                       │
│ The test files included in this distribution were all │
│ created from the plain-text file, 'csvsource.txt'     │
│ using the --create option. Study this file for more   │
│ information.                                          │
└───────────────────────────────────────────────────────┘
@end example
@end ifnothtml
@ifhtml
@cartouche
During development, this option was used to create various test files to 
exercise the functionality and general syntax checking of the application.

The test files included in this distribution were all created from 
the plain-text file, @t{'csvsource.txt'} using the @t{--create} option. 
Study this file for more information.
@end cartouche
@end ifhtml

The @t{create} option takes two or three arguments:@*
@verb{|  1) source filename|}@*
@verb{|  2) UTF encoding|} (utf8, utf8bom, utf16le, or utf16be)@*
@verb{|  3) newline format (optional) lf   |} linefeed only, or@*
@verb{|                               crlf |} carriage-return/linefeed pairs

The CSV file created through this option takes the name of the source file, 
but with the @t{".csv"} filename extension. Example:@*
@verb{|   csvsource.txt --> csvsource.csv|}@*
@verb{|   csvsource     --> csvsource.csv|}

Although the source file is technically a plain-text file, it could also 
be described as a @b{@emph{pure UTF-8}} encoded CSV file.

@Subsubheading{Exception:,.........}

The newline character, (either a linefeed @t{'\n'} or a carriage-return/linefeed 
pair @t{"\r\n"}) is a special character which signals the end of a record 
within a CSV file. However, it is fairly common to use newlines within a 
long, free-form field to indicate line breaks, so long as the field is enclosed 
within secondary-delimeter pairs.

To test this functionality, a special two-character sequence within the source 
file: @LDQ{}@b{\n}@RDQ{} is recognized by the @t{'createCsvFile'} method as 
an @b{@emph{embedded newline}} character. This allows for testing embedded 
newlines within a field.@*

Please @pxref{CSV Encoding} for additional information about newlines 
and commas embedded within a CSV record.
@ifnothtml
@sp 2
@end ifnothtml
@ifhtml
@sp 1
@end ifhtml

@verbatim
   EXAMPLES:  csvv --create=source_data.txt,utf16le
              csvv --create="source data.txt,utf16be,crlf"
              csvv --create="source data.txt, utf8bom, crlf"
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =    --capture option     =
@c = = = = = = = = = = = = = =
@cindex csv capture option
@cindex option, csv capture
@anchor{csv capture option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}capture  Capture a range of records.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;capture</span>
<span style="margin-left:2.0em;">
Capture a range of records.
</span></span>
@end html
@indentedblock
Extract a range of records from the source file, and if the records are 
UTF-16 encoded, convert them to UTF-8.

Format the data for analysis, and write the formatted data to a file in 
the current working directory.@*
The name of the target file is: @t{"@b{captured_records.txt}"}

The @t{--capture} option takes three(s) parameters:@*
@b{@verb{|  a)|}} the source filename,@*
@b{@verb{|  b)|}} the index of the first record to be captured (>=0), and@*
@b{@verb{|  c)|}} the index of the last record to be captured (>first record)@*
Note that the header record (index 0) is always captured.

During development, certain records in the source files may either be 
malformed or, @b{@emph{gasp}} the application may have parsed the record incorrectly. 
This option may be used to isolate the suspected range of records and format 
them as both text and @t{ASCII} hex, which may then be used to visually analyze 
the data. See example output below.

@verb{|   EXAMPLE:  csvv --capture="SourceRecords.csv,5,6"|}
@sp 1

@ifnothtml
@verbatim
Sample Output:
--------------
HEADER RECORD - index:00 
-------------------------
  N  A  M  E  ,  A  G  E  ,  O  C  C  U  P  A  T  I  O  N \n
 4E 41 4D 45 2C 41 47 45 2C 4F 43 43 55 50 41 54 49 4F 4E 0A

RECORD INDEX :05 
-----------------
   成   吉   思   汗  ,  9  6  2  ,   马   语   者  ,  "   蒙   古  ,
 6210 5409 601D 6C57 2C 39 36 32 2C 9A6C 8BED 8005 2C 22 8499 53E4 2C
   肯   特   山   脊  " \n
 80AF 7279 5C71 810A 22 0A
 
RECORD INDEX :06 
-----------------
  "  B  o  n  d  ,     J  a  m  e  s  "  ,  3  6  ,  w  o  m  e  n \n 
 22 42 6F 6E 64 2C 20 4A 61 6D 65 73 22 2C 33 36 2C 77 6F 6D 65 6E 0A

@end verbatim
@end ifnothtml
@ifhtml
@c Chinese characters are manually aligned.
@Subsubheading{Sample Output,.............}
@html
<div class="tscap" style="background-color:inherit; max-width:46.90em;"><span>HEADER RECORD - index:00</span>
<span>-------------------------</span>
<span>  N  A  M  E  ,  A  G  E  ,  O  C  C  U  P  A  T  I  O  N \n</span>
<span> 4E 41 4D 45 2C 41 47 45 2C 4F 43 43 55 50 41 54 49 4F 4E 0A</span>

<span>RECORD INDEX :05</span>
<span>-----------------</span>
<span class="cjk">    成    吉    思    汗   ,  9  6  2   ,    马    语    者   ,  "    蒙    古   ,</span>
<span style="font-family:monospace"> 6210 5409 601D 6C57 2C 39 36 32 2C 9A6C 8BED 8005 2C 22 8499 53E4 2C</span>
<span class="cjk">    肯    特    山    脊  " \n</span>
<span style="font-family:monospace"> 80AF 7279 5C71 810A 22 0A</span>
<span></span>
<span>RECORD INDEX :06 </span>
<span>-----------------</span>
<span>  "  B  o  n  d  ,     J  a  m  e  s  "  ,  3  6  ,  w  o  m  e  n \n </span>
<span> 22 42 6F 6E 64 2C 20 4A 61 6D 65 73 22 2C 33 36 2C 77 6F 6D 65 6E 0A</span>
<span></span>
<br></div>
@end html
@end ifhtml
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =    --stats option       =
@c = = = = = = = = = = = = = =
@cindex csv stats option
@cindex option, csv stats
@anchor{csv stats option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}stats  Report statistics for source file.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;stats</span>
<span style="margin-left:2.0em;">
Report statistics for source file.
</span></span>
@end html
@indentedblock
Display information about the specified source file.@*
This information includes basic information about the file itself, as well 
as information about the CSV data contained in the file. 

The following example illustrates the type of information reported.@*
This test file shown here is included as part of the distribution package.

@verbatim
Source File Stats
------------------------------------
Filename   : utf8bom_cr.csv            # Name of the source file
File type  : REG                       # File type (must be Regular)
File bytes : 388                       # Size of the file in bytes
Mod time   : 2024-02-15T19:09:30       # Timestamp (modification time)
Permissions: r:1 w:1                   # Read and Write permission flags
Encoding   : UTF-8bom Cr:yes Hdr:yes   # Type of character encoding
                                       # Type of newline: Linefeed only,
                                       #  or Carriage-return/Linefeed pairs
                                       # Report whether file includes a
                                       #  header record
Records    : 9        Fields : 4       # Number of records (not incl. header)
                                       # Number of fields per record


   EXAMPLE:  csvv --stats --filespec='utf8bom_cr.csv'
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =  --dimensions option    =
@c = = = = = = = = = = = = = =
@cindex csv dimensions option
@cindex option, csv dimensions
@anchor{csv dimensions option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}dimensions  Report terminal window dimensions.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;dimesions</span>
<span style="margin-left:2.0em;">
Report terminal window dimensions.
</span></span>
@end html
@indentedblock
Report the size of the terminal window expressed as the number of rows and 
columns.

When deciding how to format the CSV data to fit within the terminal window, 
the window dimensions must first be calculated. This option simply reports 
the results of that calculation.

The following example shows how the report is written to the display.@*
The report preceeds the formatted output of the CSV file data.

@verbatim
┌───────────────────────────────────────┐
│ Terminal Size: 39 rows x 154 columns  │
└───────────────────────────────────────┘
Press Any Key To Continue...



   EXAMPLES:  csvv --dimensions
              csvv --dime --filespec=Contacts.csv
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =   --terminfo option     =
@c = = = = = = = = = = = = = =
@cindex csv terminfo option
@cindex option, csv terminfo
@anchor{csv terminfo option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}terminfo  Report terminal configuration.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;terminfo</span>
<span style="margin-left:2.0em;">
Report terminal configuration.
</span></span>
@end html
@indentedblock
Used during development, this option reports the current terminal window 
configuration data as reported by the @t{AnsiCmd} link library. 
@ifnothtml
@*
@end ifnothtml
Please @pxref{acGetTermInfo} for further information about terminal configuration.

@verbatim
Terminal Configuration Flags
---------------------------------------------
istrip : 7-bit input                 disabled
opost  : output flags                enabled 
onlcr  : convert newlines to cr/lf   enabled 
xtabs  : convert tabs to spaces      disabled
cread  : read terminal input         enabled 
csize  : character width             8 bits
icanon : enable canonical input      disabled
echo   : echo character input        disabled
echonl : echo newlines               disabled
echoctl: echo control chars as (^x)  enabled 
iexten : posix extensions            enabled 


   EXAMPLE:  csvv --terminfo
@end verbatim
@end indentedblock
@sp 1


@c = = = = = = = = = = = = = =
@c =    --pause option       =
@c = = = = = = = = = = = = = =
@cindex csv pause option
@cindex option, csv pause
@anchor{csv pause option}
@ifnothtml
@sp 1
@w{ @minus{}@minus{}pause  Pause before display of formatted data.}

@end ifnothtml
@html
<span style="font-size:110%;"><span style="font-weight:bolder;"> &ndash;&hairsp;&ndash;pause</span>
<span style="margin-left:2.0em;">
Pause before display of formatted data.
</span></span>
@end html
@indentedblock
This option is included primarily for use during development.@*
When debugging output is enabled, it is written to the display during startup 
and during analysis of the source file. This option causes the program to wait 
for a keypress before the debugging information is overwritten by the formatted 
CSV records.

@verb{|   EXAMPLE:  csvv --pause --filespec='Contacts.csv'|}
@end indentedblock
@sp 2


@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@c Chapter 08.03 CSV Encoding  - - - - - - - - - - - - - - - - - - - - - - - - -
@c               Examples of CSV-encoding formats  - - - - - - - - - - - - - - -
@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@node CSV Encoding
@unnumbered CSV Encoding
@cindex 08.03 CSV Encoding
@cindex csv encoding
@cindex encoding, csv

This chapter provides a practical, (but not official) description of 
Comma-Seperated-Values (CSV) formatting and example data for each of the 
recognized UTF encodings.

@enumerate 1
@item A CSV file consists of a list of @t{"@b{records}"}, one record per line 
of the file. Any reasonable number of records is supported.@*
Technical Note: This application uses 16-bit indices, so the practical record 
limit is 32,768 data records.

@item Each record consists of a series of @t{"@b{fields}"} seperated by the 
@t{ASCII} comma: @t{',' (2C hex)}.
@enumerate a
@item Each record must have the same number of fields.
@item Any reasonable number of fields is allowed.
@item A field may contain any reasonable number of characters, (including 
zero characters).
@item A field may contain any text character @b{@emph{except}} commas and 
newline characters, (but see note about secondary delimeters, below).
@item The first field of a record @b{@emph{does not}} have a leading comma.
@item The last field of a record @b{@emph{does not}} have a trailing comma.
@end enumerate

@item Each line is terminated by a newline. For purposes of this discussion, 
a @t{"newline"} is either a single linefeed character: @t{'\n'} or a 
carriage-return/linefeed pair: @t{"\r\n"}.

In UTF-8 encoded data the newline is represented as:@*
@verb{|   0A   or   0D0A   hex|}@*
@verb{|   \n   or   \r\n|}

In UTF-16LE (little-endian) encoded data the newline is represented as:@*
@verb{|   0A00   or   0D000A00   hex|}@*
@verb{|   \n\0   or   \r\0\n\0|}

In UTF-16BE (big-endian) encoded data the newline is represented as:@*
@verb{|   000A   or   000D000A   hex|}@*
@verb{|   \0\n   or   \0\r\0\n|}

@cindex secondary delimeters
@cindex delimeters, secondary
@anchor{Secondary Field Delimeters}
@item Secondary Field Delimeters:@*
The double-quotation character: @t{'"' (22 hex)} may be used in conjunction 
with the basic comma delimeter to create pairs of secondary field delimeters. 
This is useful (and necessary) if the field contains either commas or newlines 
(as defined above). An example of a record using a secondary-delimeter pair 
for the second field of the three-field record is given:@*
@verb{| Hello!,"When, where\nand why",vape?|}
@verb{| 48656C6C6F212C225768656E2C2077686572652C0A616E6420776879222C766170653F0A|}@*
@verb{|  H e l l o ! , " W h e n ,   w h e r e ,\n a n d   w h y " , v a p e ?\n|}@*
@verb{|              └──        ┴┴              ┴┴               ──┘|}@*
@verb{|          opening        embedded        embedded     closing|}@*
@verb{|        delimeter        comma           newline    delimeter|}

The secondary delimeters enclose the data of the field: @verb{|,".....",|} 
and have the effect of dereferencing those special characters, which are then 
interpreted as plain text within the field.
@end enumerate

@Subsubheading{A Note On Creating CSV Files With Secondary Delimeters,
               ......................................................}
               
The CsvView application includes the functionality to create CSV files 
from a plain-text source file. (@pxref{csv create option}) 
The CSV file is created using the desired UTF encoding and newline format.

Commas and newlines within a CSV file are interpreted as field-delimeter and 
end-of-record, respectively. However, these characters may be interpreted 
as ordinary text @b{@emph{IF}} they are enclosed within secondary delimeters 
as described above.

Within the source file used to create CSV output, a comma embedded within 
secondary delimeters is automatically identified during conversion; however, 
because the newline character is a @LDQ{}special@RDQ{} character 
(i.e. a control character) within a plain-text file, it must be handled as 
a special case.

The design decision was made that within a source file used to generate 
CSV files, @emph{embedded newlines} are 
designated using a sequence of two ordinary @t{ASCII} characters:@*
@b{@verb{|   backstroke: '\' plus a lower case n: 'n'|}}@*
During creation of the CSV file, this sequence is converted to a newline 
character as defined above.

Comment lines may be used within the source file and will be ignored by the 
conversion algorithm. Comments are identified by using the hash character ('#') 
as the first character of the line. When the hash character is encountered, 
the entire source line will be discarded.

@b{@i{Technical Note:}} Secondary delimeters must not be used in the first record 
of the source file (header record). This is because when scanning the first 
record of a CSV file, the type of encoding has not yet been determined.

@hr40
Technical Note: It is possible that some designers of CSV files will attempt 
to be @LDQ{}creative@RDQ{} by using a character other than the double-quote 
character to create secondary-delimeter pairs. Some examples would be the 
astrisk @t{'*'} or the semicolon @t{';'}. CsvView does not support this kind 
of @LDQ{}creativity@RDQ{}, but if you often encounter such constructs, 
the @t{'convertDoubleQuotes'} and @t{'dqDelimeterCount'} methods could be 
generalized to accomodate such constructs.
@sp 2


@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@c Chapter 08.04 Build CsvView from Source   - - - - - - - - - - - - - - - - - -
@c               Command line options  - - - - - - - - - - - - - - - - - - - - -
@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@node Build CsvView from Source
@unnumbered Build CsvView from Source
@cindex 08.04 Build CsvView from Source
@cindex build CsvView
@cindex csvview build

@Subheading{Tools,-----}

The @b{@t{CsvView}} demonstration application and the @b{@t{AnsiCmd}} library 
upon which it is built, are written in C++ and are built with the GNU C++ 
compiler and linker. No additional link libraries or third-party tools are needed.

GCC (G++) version 7 and above with support for C++17 (gnu++17) or greater.

@Subheading{Compiling,---------}

The make file, @t{'Makefile'} includes several options for building the 
application.

@example
Examples:
  gmake all          @EMDASH Execute all other options in sequence
  gmake clean        @EMDASH Delete the executable and object files
  gmake refreshlib   @EMDASH Copy the necessary files from parent directory
  gmake              @EMDASH Build out-of-date source files and re-link
@end example

@Subheading{Build Options,-------------}

The @t{AnsiCmd} library has two(2) global directives which affect conditional 
compilation of the library.@*
@verb{|  1) DEBUG_ANSICMD|} Enable or disable debugging methods and data.@*
@verb{|  2) INCLUDE_API  |} Include the high-level @t{API} methods and data.

Both of these conditional-compile directives should be set to zero ('0') 
when building the @t{AnsiCmd} library (@t{gmake buildlib}).@*
While neither directive affects the functionality of the @t{CsvView} application, 
disabling them will reduce the size of the executable by approximately 600 Kbytes.@*
Please @pxref{Build Options} for the @t{AnsiCmd} library for more information 
about conditional compilation.

@enumerate 1
@item DEBUG_CSVV @NDASH Global control of debugging code for the application.
Various condition-compilation directives for debugging are defined for 
individual methods, and when the @t{DEBUG_CSVV} directive is set to zero 
('0'), all of the following directives will also be disabled.

@enumerate a
@item DEBUG_PF @NDASH Defined in the @t{'SrcScan'} method and used to 
control code for parsing the fields of a record.
@item DEBUG_RR @NDASH Defined in the @t{'readRecord'} method, this directive 
controls debugging output for the rather complex task of reading a single 
record from a CSV-formatted file.
@item DEBUG_isUTF16 @NDASH As the name implies, this directive is defined 
in the @t{'isUTF16'} method which scans the first record of the source file 
to determine whether the file is encoded as UTF-16 data.
@item DEBUG_B2W @NDASH Defined in the @t{'convert2UTF16'} method, this directive 
reports intermediate information for conversion from UTF-8 to UTF-16 data.
@item DEBUG_CNL @NDASH Defined in the @t{'convertNewlines'} method, this 
directive reports on the process of converting linefeed characters to 
carriage-return/linefeed pairs.
@item DEBUG_DNL @NDASH Defined in the @t{'deflateNewlines'} method which 
complements the @t{'convertNewlines'} method by converting carriage-return/linefeed 
pairs to linefeed characters. 
@item DEBUG_isUTF8b @NDASH Defined in the @t{'isUTF8bom'} method, this directive 
reports the intermediate steps in determining whether the source CSV file 
contains a UTF-8 byte-order mark (BOM).
@item DEBUG_CDQ @NDASH Defined in the @t{'convertDoubleQuotes'} method, this 
directive reports the intermediate steps in processing double-quotation-mark 
characters which are used as secondary delimeters within a field.@*
Please @pxref{Secondary Field Delimeters} for more information on secondary 
delimeters.
@item DEBUG_GCLA @NDASH Defined in the @t{'GetCommandLineArgs'} method, this 
directive enables reporting on parsing of the application's command line arguments.
@item DEBUG_VP @NDASH Defined in the @t{'DisplayData'} method, 
this directive inserts visible characters ('-') instead of whitespace for 
padding fields to their full width.
@end enumerate
@end enumerate

@Subheading{Testing the Build,-----------------}

To test the build, invoke with a request for the CsvView version number. 
You should get something similar to the following:

@example
./csvv --version

CsvView (csvv) v:0.0.06 Copyright(c) 2024-2026  The Software Samurai   
=======================================================================
AnsiCmd Link Library v:0.0.07  (gString v:0.0.37)                      
-----------------------------------------------------------------------
License GPLv3+: GNU GPL version 3 <http://gnu.org/licenses/gpl.html>
This is free software: you are free to modify and/or redistribute it
under the terms set out in the license.
There is NO WARRANTY, to the extent permitted by law.
@end example

If you have forgotten to disable the @t{AnsiCmd} library debugging code, 
and/or the @t{AnsiCmd API-level} code (see above), a warning message will 
be appended to the above output.
@spi

@c - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@raisesections
@sp 2
