cape.dkit.tsvfile: Space-separated value read/write

This module contains a basic interface in the spirit of cape.dkit.ftypes for standard comma-separated value files. It creates a class, TSVFile that does not rely on the popular numpy.loadtxt() function.

If possible, the column names (which become keys in the dict-like class) are read from the header row. If the file begins with multiple comment lines, the column names are read from the final comment before the beginning of data.

class cape.dkit.tsvfile.TSVFile(fname=None, **kw)

Class for reading space-separated files

Call:
>>> db = TSVFile(fname, **kw)
>>> db = TSVFile(f, **kw)
Inputs:
fname: str

Name of file to read

f: file

Open file handle

Outputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

db.cols: list[str]

List of columns read

db.opts: TSVFileOpts

Options for this instance

db.defns: dict[TSVFileDefn]

Definitions for each column

db[col]: np.ndarray | list

Numeric array or list of strings for each column

See also:

  • cape.dkit.ftypes.basefile.BaseFile

  • cape.dkit.ftypes.basefile.TextInterpreter

Versions:

  • 2021-01-14 @ddalle: Version 1.0

c_read_tsv(fname, **kw)

Read an entire TSV file, including header using C

Call:
>>> db.read_tsv(fname)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

fname: str

Name of file to read

See Also:
Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

c_read_tsv_data(f)

Read data portion of TSV file using C extension

Call:
>>> db.c_read_tsv_data(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

List of column names

Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

create_c_dtypes()

Initialize db._c_dtypes for C text input

Call:
>>> db.create_c_dtypes()
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

Effects:
db._c_dtypes: list[int]

List of integer codes for each data type

Versions:

  • 2019-11-29 @ddalle: Version 1.0 (CSVFile)

py_read_tsv(fname)

Read an entire TSV file with pure Python

Call:
>>> db.py_read_tsv(fname)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

fname: str

Name of file to read

See Also:
Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

py_read_tsv_data(f)

Read data portion of TSV file using Python

Call:
>>> db.py_read_tsv_data(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

List of column names

Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

read_tsv(fname)

Read a TSV file, including header

Reads either entire file or from current location

Call:
>>> db.read_tsv(f)
>>> db.read_tsv(fname)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

File open for reading

fname: str

Name of file to read

See Also:
Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

  • 2021-01-14 @ddalle: Version 1.0

read_tsv_data(f)

Read data portion of TSV file

Call:
>>> db.read_tsv_data(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

List of column names

Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

  • 2019-11-29 @ddalle: Version 1.1; try C first

  • 2021-01-14 @ddalle: Version 1.0

read_tsv_dataline(f)

Read one data line of a TSV file

Call:
>>> db.read_tsv_dataline(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

  • 2021-01-14 @ddalle: Version 1.0

read_tsv_firstrowtypes(f)

Get initial guess at data types from first data row

If (and only if) the DefaultType input is an integer type, guessed types can be integers. Otherwise the sequence of possibilities is float, complex, str.

Call:
>>> db.read_tsv_firstrowtypes(f, **kw)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

DefaultType: {"float"} | str

Name of default class

Versions:

  • 2019-11-25 @ddalle: Version 1.0 (CSVFile)

  • 2021-01-14 @ddalle: Version 1.0

read_tsv_header(f)

Read column names from beginning of open file

Call:
>>> db.read_tsv_header(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

List of column names

Versions:

  • 2019-11-12 @ddalle: Version 1.0 (CSVFile)

read_tsv_headerdefaultcols(f)

Create column names “col1”, “col2”, etc. if needed

Call:
>>> db.read_tsv_headerdefaultcols(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

If not previously determined, this becomes ["col1", "col2", ...] based on number of columns in the first data row

Versions:

  • 2019-11-27 @ddalle: Version 1.0

read_tsv_headerline(f)

Read line and process column names if possible

Call:
>>> db.read_tsv_headerline(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: None | list[str]

List of column names if read

db._tsv_header_once: True | False

Set to True if column names are read at all

db._tsv_header_complete: True | False

Set to True if next line is expected to be data

Versions:

  • 2019-11-22 @ddalle: Version 1.0 (CSVFile)

write_tsv(fname, cols=None, **kw)

Write a comma-separated file of some of the coefficients

Call:
>>> db.write_tsv(fname, cols=None, **kw)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

fname: str

Name of TSV file to write

cols: {None} | list (str)

List of coefficients to write, or write all coefficients

fmt: {None} | str

Format string to be used for each row (optional)

fmts: dict | str

Dictionary of formats to use for each coeff

comments: {"#"} | str

Comment character, used as first character of file

delim: {", "} | str

Delimiter

translators: {{}} | dict

Dictionary of col name translations, e.g. CAF -> CA; this dictionary is run in reverse

Versions:

  • 2018-06-11 @ddalle: Version 1.0 (cape.dkit.db.db1)

  • 2020-01-15 @jmeeroff: Version 1.0

  • 2020-04-01 @ddalle: Version 2.0; full options

write_tsv_dense(fname=None, cols=None)

Write dense TSV file using WriteFlag for each column

Call:
>>> db.write_tsv_dense(f, cols=None)
>>> db.write_tsv_dense(fname=None, cols=None)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

File open for writing

fname: {db.fname} | str

Name of file to write

cols: {db.cols} | list[str]

List of columns to write

Versions:

  • 2019-12-05 @ddalle: Version 1.0 (CSVFile)

class cape.dkit.tsvfile.TSVFileDefn(_optsdict=None, _warnmode=1, **kw)
class cape.dkit.tsvfile.TSVFileOpts(_optsdict=None, _warnmode=1, **kw)
class cape.dkit.tsvfile.TSVSimple(fname=None, **kw)

Class to read TSV file with only float data

This class differs from TSVFile in that it is less flexible, does not permit multirow or empty headers, has fixed delimiter and comment characters, and assumes all data is a float with the system default length.

Call:
>>> db = TSVSimple(fname, **kw)
Inputs:
fname: str

Name of file to read

Outputs:
db: cape.dkit.ftypes.tsvfile.TSVSimple

TSV file interface

db.cols: list[str]

List of columns read

db.opts: dict

Options for this instance

db.opts[“Definitions”]: dict

Definitions for each column

db[col]: np.ndarray | list

Numeric array or list of strings for each column

See also:

  • cape.dkit.ftypes.basefile.BaseFile

Versions:

  • 2019-11-26 @ddalle: Started

read_tsvsimple(fname)

Read an entire TSV file, including header

The TSV file requires exactly one header row, which is the first non-empty line, whether or not it begins with a comment character (which must be "#"). All entries, both in the header and in the data, must be separated by a ,.

Call:
>>> db.read_tsvsimple(fname)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVSimple

TSV file interface

fname: str

Name of file to read

See Also:
Versions:

  • 2019-11-27 @ddalle: Version 1.0 (CSVSimle)

read_tsvsimple_data(f)

Read data portion of simple TSV file

Call:
>>> db.read_tsvsimple_data(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVSimple

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

List of column names

Versions:

  • 2019-11-25 @ddalle: Version 1.0

  • 2021-07-09 @ddalle: Version 2.0; np.fromfile()

read_tsvsimple_header(f)

Read column names from beginning of open file

Call:
>>> db.read_tsvsimple_header(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

List of column names

Versions:

  • 2019-11-12 @ddalle: Version 1.0 (CSVSimple)

  • 2021-01-14 @ddalle: Version 1.0

  • 2021-07-09 @ddalle: Version 2.0

    • Allow multiple comments

    • Allow no header

read_tsvsimple_headerdefaultcols(f)

Create column names “col1”, “col2”, etc. if needed

Call:
>>> db.read_tsvsimple_headerdefaultcols(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: list[str]

If not previously determined, this becomes ["col1", "col2", ...] based on number of columns in the first data row

Versions:

  • 2019-11-27 @ddalle: Version 1.0 (TSVFile)

  • 2021-07-09 @ddalle: Version 1.0

read_tsvsimple_headerline(f)

Read line and process column names if possible

Call:
>>> db.read_tsvsimple_headerline(f)
Inputs:
db: cape.dkit.ftypes.tsvfile.TSVFile

TSV file interface

f: file

Open file handle

Effects:
db.cols: None | list[str]

List of column names if read

db._tsv_header_once: True | False

Set to True if column names are read at all

db._tsv_header_complete: True | False

Set to True if next line is expected to be data

Versions:

  • 2019-11-22 @ddalle: Version 1.0 (CSVFile)

  • 2021-07-09 @ddalle: Version 1.0

class cape.dkit.tsvfile.TSVTecDatFile(fname=None, **kw)
read_tsvtecdat(fname: str)

Read an entire TSV file, including header

The TSV file requires exactly one header row, which is the first non-empty line, whether or not it begins with a comment character (which must be "#"). All entries, both in the header and in the data, must be separated by a ,.

Call:
>>> db.read_tsvtecdat(fname)
Inputs:
db: TSVTecDatFile

TSV file interface

fname: str

Name of file to read

read_tsvtecdat_header(fp: IOBase)

Read column names from beginning of open file

Call:
>>> db.read_tsvtecdat_header(fp)
Inputs:
db: TSVTecDatFile

TSV file interface

fp: IOBase

Open file handle

Effects:
db.cols: list[str]

List of column names

Versions:

  • 2024-01-23 @ddalle: v1.0