cape.attdb.ftypes.basedata: Common ATTDB data container

This module provides the class BaseData as a subclass of dict that contains methods common to each of the other data-like file readers and writers. It defines common attributes that each other such data-file class will have and provides several other common methods.

For example, it defines a __repr__() method that updates with the name of any other classes subclassed to this one.

Finally, having this common template class provides a single point of entry for testing if an object is based on a product of the cape.attdb.ftypes module. The following Python sample tests if any Python object db is an instance of any class from this data-file collection.

isinstance(db, cape.attdb.ftypes.BaseData)
class cape.attdb.ftypes.basedata.BaseData(**kw)

Generic class for storing data from a data-style file

This class has no initialization method, and as such it is unlikely that there will be instances of this class in use. It provides methods and structure to other classes.

This class inherits from dict and can be used in that matter in the unlikely event that it’s useful.

Outputs:
db: cape.attdb.ftypes.csv.CSVFile

CSV file interface

db.cols: list[str]

List of columns read

db.opts: BaseDataOpts | db._optscls

Options for this instance

db.defns: dict[BaseDataDefn]

Definitions for each data column

db[col]: np.ndarray | list

Numeric array or list of strings for column col

See also:
Versions:

  • 2019-11-26 @ddalle: Version 1.0

  • 2020-02-02 @ddalle: Second version

apply_defn_tag(col, tagdef=None)

Apply default Tag to each definition

Call:
>>> db.apply_defn_tag(col, tagdef=None)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column for which to set default tag

tagdef: {None} | str

Manually specified default tag

Versions:

  • 2020-03-18 @ddalle: Version 1.0

apply_defns_tag(cols=None)

Apply all default Tag properties based on col name

Call:
>>> db.apply_defns_tag(cols=None)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

cols: list[str]

List of column names

Versions:

  • 2020-03-18 @ddalle: Version 1.0

burst_col(col)

Remove a column and its definition is possible

Call:
>>> V = db.burst_col(col)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column

Outputs:
V: np.ndarray | list | scalar

Value(s) to save for specified column

Versions:

  • 2020-03-19 @ddalle: Version 1.0

create_defn(col, V, **kw)

Create and save a new definition based on values

Call:
>>> defn = db.create_defn(col, V, **kw)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column (used for “Tag” option)

V: list | np.ndarray

Values for column col

kw: dict

Optional overrides or additions to definition

Outputs:
defn: db._defncls

Definition based on values V

Effects:

db[col]: defn

Versions:

  • 2020-03-19 @ddalle: Version 1.0

classmethod create_tagcols()

Invert cls._tagmap as cls._tagcols

Call:
>>> cls.create_tagcols()
Inputs:
cls: type

Data container class

Versions:

  • 2020-03-19 @ddalle: Version 1.0

finish_defn(col)

Apply any defaults to a data column definition

This first checks instance options like "DefaultType" and then the global defaults such as defn._rc["Type"].

Call:
>>> db.finish_defn(col)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Data column name

Versions:
finish_defns(cols=None)

Apply any defaults to data column definitions

This first checks instance options like "DefaultType" and then the global defaults such as defn._rc["Type"].

Call:
>>> db.finish_defns(cols=None)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

cols: list[str]

List of column names

Versions:

  • 2020-02-03 @ddalle: Version 1.0

  • 2020-03-31 @ddalle: Handled db.opts properly

genr8_defn(col, V, **kw)

Generate a new definition based on values

Call:
>>> defn = db.genr8_defn(col, V, **kw)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column (used for “Tag” option)

V: list | np.ndarray

Values for column col

kw: dict

Optional overrides or additions to definition

Outputs:
defn: db._defncls

Definition based on values V

Versions:

  • 2020-03-19 @ddalle: Version 1.0

get_col_by_tag(tag, coldef=None)

Return the first col with specified “Tag”, if any

Call:
>>> col = db.get_col_by_tag(tag)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

tag: str

Target “Tag” from column definitions

coldef: {None} | str

Default column name

Outputs:
col: None | str

Name of column for which to set default tag

Versions:

  • 2020-03-18 @ddalle: Version 1.0

get_col_dtype(col)

Get data type for specific column

Call:
>>> cls = db.get_col_type(col, prop)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column

Outputs:
cls: "int" | "float" | "str" | str

Name of data type

Versions:

  • 2019-11-24 @ddalle: Version 1.0

get_col_prop(col, prop, vdef=None)

Get property for specific column

Call:
>>> v = db.get_col_prop(col, prop, vdef=None)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column

prop: str

Name of property

vdef: {None} | any

Default value if not specified in db.opts

Outputs:
v: any

Value of defns[col][prop] if possible; defaulting to defns["_"][prop] or vdef

Versions:

  • 2019-11-24 @ddalle: Version 1.0

  • 2019-12-31 @ddalle: Moved from basefile

  • 2020-02-01 @ddalle: Using BaseDataDefn

get_col_type(col)

Get data type for specific column

Call:
>>> cls = db.get_col_type(col, prop)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column

Outputs:
cls: "int" | "float" | "str" | str

Name of data type

Versions:

  • 2019-11-24 @ddalle: Version 1.0

get_cols_by_tag(tag)

Return all col with specified “Tag”

Call:
>>> cols = db.get_cols_by_tag(tag)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

tag: str

Target “Tag” from column definitions

Outputs:
cols: list[str]

Name of column for which to set default tag

Versions:

  • 2020-03-18 @ddalle: Version 1.0

get_defn(col)

Get column definition for data column col

Call:
>>> defn = db.get_defn(col)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column

Outputs:
defn: BaseDataDefn

Definition for column col

Versions:

  • 2020-02-01 @ddalle: Version 1.0

get_defns()

Get dictionary of column definitions

Call:
>>> defns = db.get_defns()
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

Outputs:
defns: dict[BaseDataDefn]

Definitions for each column

Versions:

  • 2019-12-31 @ddalle: Version 1.0

  • 2020-02-01 @ddalle: Move from opts["Definitions"]

get_option(key, vdef=None)

Get an option, appealing to default if necessary

Call:
>>> val = db.get_option(key, vdef=None)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

key: str

Name of option to access

vdef: {None} | any

Default option for fallback

Outputs:
val: db.opts[key] | db.opts._rc[key] | vdef

Value of option with fallback

Versions:

  • 2019-12-31 @ddalle: Version 1.0

  • 2020-02-01 @ddalle: Using BaseDataOpts

keeponly_cols(cols)

Remove all columns outside specified list

Call:
>>> db.keeponly_cols(cols)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

cols: list | set

Name of columns to keep

Versions:

  • 2020-12-21 @ddalle: Version 1.0

make_defn(col, V, **kw)

Access or create new definition based on values

Call:
>>> defn = db.make_defn(col, V, **kw)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column (used for “Tag” option)

V: list | np.ndarray

Values for column col

kw: dict

Optional overrides or additions to definition

Outputs:
defn: db._defncls

Definition based on values V

Effects:

db[col]: defn

Versions:

  • 2020-03-19 @ddalle: Version 1.0

  • 2020-06-24 @ddalle: Version 1.1; merge defns

process_kw(**kw)

Process options from keyword arguments

Call:
>>> opts = db.process_kw(**kw)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

kw: dict

Arbitrary keyword arguments

Outputs:
opts: BaseDataOpts | db._optscls

Validated options from kw

Versions:

  • 2020-02-02 @ddalle: Version 1.0

process_kw_values()

Process Values argument for manual column values

Call:
>>> db.process_kw_values()
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

Options:
Values: dict

Dictionary of values for some columns

ExpandScalars: True | False

Option to expand scalars to match dimension of arrays

Versions:

  • 2019-11-12 @ddalle: v1.0

  • 2019-12-31 @ddalle: v1.1; Remove pop() and output

  • 2020-02-02 @ddalle: v1.2; Delete kw as input

  • 2023-02-23 @ddalle: v1.3; ExpandScalars -> False

rename_col(col1, col2)

Rename a column col1 to col2

Call:
>>> db.rename_col(col1, col2)
Inputs:
db: cape.attdb.ftypes.basefile.BaseFile

Data file interface

col1: str

Existing column name

col2: str

New column name

Versions:

  • 2021-07-09 @ddalle: Version 1.0

save_col(col, v)

Save a column value, updating other metadata as needed

Call:
>>> db.save_col(col, v)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column

v: np.ndarray | list | scalar

Value(s) to save for specified column

Versions:

  • 2019-11-12 @ddalle: Started

  • 2020-02-14 @ddalle: Tweak rules for cols append

set_col_prop(col, prop, v)

Set property for specific column

Call:
>>> v = db.set_col_prop(col, prop, v, vdef=None)
Inputs:
db: cape.attdb.ftypes.basedata.BaseData

Data container

col: str

Name of column

prop: str

Name of property

v: any

Value to set to defns[col][prop]

Versions:

  • 2021-01-22 @aburkhea: Version 1.0

class cape.attdb.ftypes.basedata.BaseDataDefn(_optsdict=None, _warnmode=1, **kw)
finish()

Apply default values from defn._rc

Call:
>>> defn.finish()
Inputs:
defn: BaseDataDefn

Data column definition

Versions:

  • 2020-01-31 @ddalle: Version 1.0

  • 2020-03-06 @ddalle: Rename apply_defaults()

get_dtype()

Get (and save) data type (DType) based on Type

Call:
>>> dtype = defn.get_dtype()
Inputs:
defn: BaseDataDefn

Data column definition

Outputs:
dtype: str

Data type, looks up Type in defn._dtypemap

Versions:

  • 2020-02-01 @ddalle: Version 1.0

class cape.attdb.ftypes.basedata.BaseDataOpts(_optsdict=None, _warnmode=1, **kw)
finish_defn(defn)

Apply any defaults to a data column definition

This first checks instance options like "DefaultType" and then the global defaults such as defn._rc["Type"].

Call:
>>> opts.finish_defn(defn)
Inputs:
opts: BaseDataOpts

Options interface for cape.attdb.ftypes

defn: BaseDataDefn | opts._defncls

Data column definition

Versions:

  • 2020-01-31 @ddalle: Version 1.0

get_defn(col)

Get the processed definition, without applying defaults

This method returns a definition-type instance that checks the Definition option and dict-like options like Types. Defaults, both from the definition class’s _rc attribute and options like DefaultType, are not applied so that they can be automatically guessed from the data.

Call:
>>> defn = opts.get_defn(col)
Inputs:
opts: BaseDataOpts

Options interface for cape.attdb.ftypes

col: str

Name of data column

Outputs:
defn: BaseDataDefn | opts._defncls

Column definition for col

Versions:

  • 2020-01-31 @ddalle: Version 1.0

classmethod set_defncls(defncls)

Add all the “default” options from a definition class

This loops through the options in defncls._optlist and adds two versions of them to cls._optlist:

  • Prefixed with “Default”

  • Suffixed with “s”

For example if defncls._optlist has "Type", this adds "DefaultType" and "Types" to cls._optlist.

Call:
>>> cls.set_defncls(cls, defncls)
Inputs:
cls: BaseDataOpts

Parent options class

defncls: BaseDataDefn

Definition options class

Versions:

  • 2020-01-30 @ddalle: Version 1.0