`cape.dkit.basedata`: Common ATTDB data container¶

This module provides the class BaseData as a subclass of dict that contains methods common to each of the other data-like file readers and writers. It defines common attributes that each other such data-file class will have and provides several other common methods.

For example, it defines a __repr__() method that updates with the name of any other classes subclassed to this one.

Finally, having this common template class provides a single point of entry for testing if an object is based on a product of the cape.dkit.ftypes module. The following Python sample tests if any Python object db is an instance of any class from this data-file collection.

isinstance(db, cape.dkit.ftypes.BaseData)

class cape.dkit.basedata.BaseData(**kw)¶

Generic class for storing data from a data-style file

This class has no initialization method, and as such it is unlikely that there will be instances of this class in use. It provides methods and structure to other classes.

This class inherits from dict and can be used in that matter in the unlikely event that it’s useful.

Outputs:

db: cape.dkit.ftypes.csv.CSVFile: CSV file interface
db.cols: list[str]: List of columns read
db.opts: BaseDataOpts | db._optscls: Options for this instance
db.defns: dict[BaseDataDefn]: Definitions for each data column
db[col]: np.ndarray | list: Numeric array or list of strings for column col

See also:

cape.dkit.ftypes.csv.CSVFile
cape.dkit.ftypes.csv.CSVSimple
cape.dkit.ftypes.textdata.TextDataFile

Versions:

2019-11-26 @ddalle: Version 1.0
2020-02-02 @ddalle: Second version

apply_defn_tag(col, tagdef=None)¶

Apply default Tag to each definition

Call:

>>> db.apply_defn_tag(col, tagdef=None)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column for which to set default tag
tagdef: {None} | str: Manually specified default tag

Versions:

2020-03-18 @ddalle: Version 1.0

apply_defns_tag(cols=None)¶

Apply all default Tag properties based on col name

Call:

>>> db.apply_defns_tag(cols=None)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
cols: list[str]: List of column names

Versions:

2020-03-18 @ddalle: Version 1.0

burst_col(col)¶

Remove a column and its definition is possible

Call:

>>> V = db.burst_col(col)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column

Outputs:

V: np.ndarray | list | scalar: Value(s) to save for specified column

Versions:

2020-03-19 @ddalle: Version 1.0

create_defn(col, V, **kw)¶

Create and save a new definition based on values

Call:

>>> defn = db.create_defn(col, V, **kw)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column (used for “Tag” option)
V: list | np.ndarray: Values for column col
kw: dict: Optional overrides or additions to definition

Outputs:

defn: db._defncls: Definition based on values V

Effects:

db[col]: defn

Versions:

2020-03-19 @ddalle: Version 1.0

classmethod create_tagcols()¶

Invert cls._tagmap as cls._tagcols

Call:

>>> cls.create_tagcols()

Inputs:

cls: type: Data container class

Versions:

2020-03-19 @ddalle: Version 1.0

finish_defn(col)¶

Apply any defaults to a data column definition

This first checks instance options like "DefaultType" and then the global defaults such as defn._rc["Type"].

Call:

>>> db.finish_defn(col)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Data column name

Versions:

2020-03-31 @ddalle: Split from finish_defns()

finish_defns(cols=None)¶

Apply any defaults to data column definitions

This first checks instance options like "DefaultType" and then the global defaults such as defn._rc["Type"].

Call:

>>> db.finish_defns(cols=None)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
cols: list[str]: List of column names

Versions:

2020-02-03 @ddalle: Version 1.0
2020-03-31 @ddalle: Handled db.opts properly

genr8_defn(col, V, **kw)¶

Generate a new definition based on values

Call:

>>> defn = db.genr8_defn(col, V, **kw)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column (used for “Tag” option)
V: list | np.ndarray: Values for column col
kw: dict: Optional overrides or additions to definition

Outputs:

defn: db._defncls: Definition based on values V

Versions:

2020-03-19 @ddalle: Version 1.0

get_col_by_tag(tag, coldef=None)¶

Return the first col with specified “Tag”, if any

Call:

>>> col = db.get_col_by_tag(tag)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
tag: str: Target “Tag” from column definitions
coldef: {None} | str: Default column name

Outputs:

col: None | str: Name of column for which to set default tag

Versions:

2020-03-18 @ddalle: Version 1.0

get_col_dtype(col)¶

Get data type for specific column

Call:

>>> cls = db.get_col_type(col, prop)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column

Outputs:

cls: "int" | "float" | "str" | str: Name of data type

Versions:

2019-11-24 @ddalle: Version 1.0

get_col_prop(col, prop, vdef=None)¶

Get property for specific column

Call:

>>> v = db.get_col_prop(col, prop, vdef=None)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column
prop: str: Name of property
vdef: {None} | any: Default value if not specified in db.opts

Outputs:

v: any: Value of defns[col][prop] if possible; defaulting to defns["_"][prop] or vdef

Versions:

2019-11-24 @ddalle: Version 1.0
2019-12-31 @ddalle: Moved from basefile
2020-02-01 @ddalle: Using BaseDataDefn

get_col_type(col)¶

Get data type for specific column

Call:

>>> cls = db.get_col_type(col, prop)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column

Outputs:

cls: "int" | "float" | "str" | str: Name of data type

Versions:

2019-11-24 @ddalle: Version 1.0

get_cols_by_tag(tag)¶

Return all col with specified “Tag”

Call:

>>> cols = db.get_cols_by_tag(tag)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
tag: str: Target “Tag” from column definitions

Outputs:

cols: list[str]: Name of column for which to set default tag

Versions:

2020-03-18 @ddalle: Version 1.0

get_defn(col)¶

Get column definition for data column col

Call:

>>> defn = db.get_defn(col)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column

Outputs:

defn: BaseDataDefn: Definition for column col

Versions:

2020-02-01 @ddalle: Version 1.0

get_defns()¶

Get dictionary of column definitions

Call:

>>> defns = db.get_defns()

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container

Outputs:

defns: dict[BaseDataDefn]: Definitions for each column

Versions:

2019-12-31 @ddalle: Version 1.0
2020-02-01 @ddalle: Move from opts["Definitions"]

get_option(key, vdef=None)¶

Get an option, appealing to default if necessary

Call:

>>> val = db.get_option(key, vdef=None)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
key: str: Name of option to access
vdef: {None} | any: Default option for fallback

Outputs:

val: db.opts[key] | db.opts._rc[key] | vdef: Value of option with fallback

Versions:

2019-12-31 @ddalle: Version 1.0
2020-02-01 @ddalle: Using BaseDataOpts

keeponly_cols(cols)¶

Remove all columns outside specified list

Call:

>>> db.keeponly_cols(cols)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
cols: list | set: Name of columns to keep

Versions:

2020-12-21 @ddalle: Version 1.0

make_defn(col, V, **kw)¶

Access or create new definition based on values

Call:

>>> defn = db.make_defn(col, V, **kw)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column (used for “Tag” option)
V: list | np.ndarray: Values for column col
kw: dict: Optional overrides or additions to definition

Outputs:

defn: db._defncls: Definition based on values V

Effects:

db[col]: defn

Versions:

2020-03-19 @ddalle: Version 1.0
2020-06-24 @ddalle: Version 1.1; merge defns

process_kw(**kw)¶

Process options from keyword arguments

Call:

>>> opts = db.process_kw(**kw)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
kw: dict: Arbitrary keyword arguments

Outputs:

opts: BaseDataOpts | db._optscls: Validated options from kw

Versions:

2020-02-02 @ddalle: Version 1.0

process_kw_values()¶

Process Values argument for manual column values

Call:

>>> db.process_kw_values()

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container

Options:

Values: dict: Dictionary of values for some columns
ExpandScalars: True | False: Option to expand scalars to match dimension of arrays

Versions:

2019-11-12 @ddalle: v1.0
2019-12-31 @ddalle: v1.1; Remove pop() and output
2020-02-02 @ddalle: v1.2; Delete kw as input
2023-02-23 @ddalle: v1.3; ExpandScalars -> False

rename_col(col1, col2)¶

Rename a column col1 to col2

Call:

>>> db.rename_col(col1, col2)

Inputs:

db: cape.dkit.ftypes.basefile.BaseFile: Data file interface
col1: str: Existing column name
col2: str: New column name

Versions:

2021-07-09 @ddalle: Version 1.0

save_col(col, v)¶

Save a column value, updating other metadata as needed

Call:

>>> db.save_col(col, v)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column
v: np.ndarray | list | scalar: Value(s) to save for specified column

Versions:

2019-11-12 @ddalle: Started
2020-02-14 @ddalle: Tweak rules for cols append

set_col_prop(col, prop, v)¶

Set property for specific column

Call:

>>> v = db.set_col_prop(col, prop, v, vdef=None)

Inputs:

db: cape.dkit.ftypes.basedata.BaseData: Data container
col: str: Name of column
prop: str: Name of property
v: any: Value to set to defns[col][prop]

Versions:

2021-01-22 @aburkhea: Version 1.0

class cape.dkit.basedata.BaseDataDefn(_optsdict=None, _warnmode=1, **kw)¶

finish()¶

Apply default values from defn._rc

Call:

>>> defn.finish()

Inputs:

defn: BaseDataDefn: Data column definition

Versions:

2020-01-31 @ddalle: Version 1.0
2020-03-06 @ddalle: Rename apply_defaults()

get_dtype()¶

Get (and save) data type (DType) based on Type

Call:

>>> dtype = defn.get_dtype()

Inputs:

defn: BaseDataDefn: Data column definition

Outputs:

dtype: str: Data type, looks up Type in defn._dtypemap

Versions:

2020-02-01 @ddalle: Version 1.0

class cape.dkit.basedata.BaseDataOpts(_optsdict=None, _warnmode=1, **kw)¶

finish_defn(defn)¶

Apply any defaults to a data column definition

This first checks instance options like "DefaultType" and then the global defaults such as defn._rc["Type"].

Call:

>>> opts.finish_defn(defn)

Inputs:

opts: BaseDataOpts: Options interface for cape.dkit.ftypes
defn: BaseDataDefn | opts._defncls: Data column definition

Versions:

2020-01-31 @ddalle: Version 1.0

get_defn(col)¶

Get the processed definition, without applying defaults

This method returns a definition-type instance that checks the Definition option and dict-like options like Types. Defaults, both from the definition class’s _rc attribute and options like DefaultType, are not applied so that they can be automatically guessed from the data.

Call:

>>> defn = opts.get_defn(col)

Inputs:

opts: BaseDataOpts: Options interface for cape.dkit.ftypes
col: str: Name of data column

Outputs:

defn: BaseDataDefn | opts._defncls: Column definition for col

Versions:

2020-01-31 @ddalle: Version 1.0

classmethod set_defncls(defncls)¶

Add all the “default” options from a definition class

This loops through the options in defncls._optlist and adds two versions of them to cls._optlist:

Prefixed with “Default”

Suffixed with “s”

For example if defncls._optlist has "Type", this adds "DefaultType" and "Types" to cls._optlist.

Call:

>>> cls.set_defncls(cls, defncls)

Inputs:

cls: BaseDataOpts: Parent options class
defncls: BaseDataDefn: Definition options class

Versions:

2020-01-30 @ddalle: Version 1.0

`cape.dkit.basedata`: Common ATTDB data container¶

Previous topic

Next topic

This Page

cape.dkit.basedata: Common ATTDB data container¶

`cape.dkit.basedata`: Common ATTDB data container¶