Brightway2-data#
Configuration#
The configuration for brightway2 is implemented as a singleton class that is created when brightway2
is imported.
- class bw2data.configuration.Config#
A singleton that stores configuration settings
- property biosphere#
Get name for
biosphere
database from user preferences.Default name is
biosphere3
; change this by changingconfig.p["biosphere_database"]
.
- property global_location#
Get name for global location from user preferences.
Default name is
GLO
; change this by changingconfig.p["global_location"]
.
Projects#
- class bw2data.project.ProjectManager#
- copy_project(new_name, switch=True)#
Copy current project to a new project named
new_name
. Ifswitch
, switch to new project.
- delete_project(name=None, delete_dir=False)#
Delete project
name
, or the current project.name
is the project to delete. Ifname
is not provided, delete the current project.By default, the underlying project directory is not deleted; only the project name is removed from the list of active projects. If
delete_dir
isTrue
, then also delete the project directory.If deleting the current project, this function sets the current directory to
default
if it exists, or to a random project.Returns the current project.
- property output_dir#
Get directory for output files.
Uses environment variable
BRIGHTWAY2_OUTPUT_DIR
;preferences['output_dir']
; or directoryoutput
in current project.Returns output directory path.
- purge_deleted_directories()#
Delete project directories for projects which are no longer registered.
Returns number of directories deleted.
- report()#
Give a report on current projects, including installed databases and file sizes.
Returns tuples of
(project name, number of databases, size of all databases (GB))
.
- request_directory(name)#
Return the absolute path to the subdirectory
dirname
, creating it if necessary.Returns
False
if directory can’t be created.
Base classes for metadata#
Serialized Dictionary#
- class bw2data.serialization.SerializedDict(dirpath=None)#
Base class for dictionary that can be serialized to or unserialized from disk. Uses JSON as its storage format. Has most of the methods of a dictionary.
Upon instantiation, the serialized dictionary is read from disk.
- backup()#
Write a backup version of the data to the
backups
directory.
- deserialize()#
Load the serialized data. Can be replaced with other serialization formats.
- flush()#
Serialize the current data to disk.
- keys() a set-like object providing a view on D's keys #
- property list#
List the keys of the dictionary. This is a property, and does not need to be called.
- load()#
Load the serialized data. Creates the file if not yet present.
- pack(data)#
Transform the data, if necessary. Needed because JSON must have strings as dictionary keys.
- random()#
Return a random key.
- serialize(filepath=None)#
Method to do the actual serialization. Can be replaced with other serialization formats.
- Args:
filepath (str, optional): Provide an alternate filepath (e.g. for backup).
- unpack(data)#
Return serialized data to true form.
- values() an object providing a view on D's values #
Compound JSON dictionary#
JSON hash tables don’t support keys like ("biosphere", "an emission")
, so the pack
and unpack
methods are used to transform data from Python to JSON and back.
Pickled Dictionary#
- class bw2data.serialization.PickledDict(dirpath=None)#
Subclass of
SerializedDict
that uses the pickle format instead of JSON.- deserialize()#
Load the serialized data. Can be replaced with other serialization formats.
- serialize()#
Method to do the actual serialization. Can be replaced with other serialization formats.
- Args:
filepath (str, optional): Provide an alternate filepath (e.g. for backup).
Metadata stores#
databases#
- class bw2data.meta.Databases(dirpath=None)#
A dictionary for database metadata. This class includes methods to manage database versions. File data is saved in
databases.json
.- backup()#
Write a backup version of the data to the
backups
directory.
- clear() None. Remove all items from D. #
- deserialize()#
Load the serialized data. Can be replaced with other serialization formats.
- flush()#
Serialize the current data to disk.
- get(k[, d]) D[k] if k in D, else d. d defaults to None. #
- increment_version(database, number=None)#
Increment the
database
version. Returns the new version.
- items() a set-like object providing a view on D's items #
- keys() a set-like object providing a view on D's keys #
- property list#
List the keys of the dictionary. This is a property, and does not need to be called.
- load()#
Load the serialized data. Creates the file if not yet present.
- pack(data)#
Transform the data, if necessary. Needed because JSON must have strings as dictionary keys.
- pop(k[, d]) v, remove specified key and return the corresponding value. #
If key is not found, d is returned if given, otherwise KeyError is raised.
- popitem() (k, v), remove and return some (key, value) pair #
as a 2-tuple; but raise KeyError if D is empty.
- random()#
Return a random key.
- serialize(filepath=None)#
Method to do the actual serialization. Can be replaced with other serialization formats.
- Args:
filepath (str, optional): Provide an alternate filepath (e.g. for backup).
- setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D #
- unpack(data)#
Return serialized data to true form.
- update([E, ]**F) None. Update D from mapping/iterable E and F. #
If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
- values() an object providing a view on D's values #
- version(database)#
Return the
database
version
methods#
- class bw2data.meta.Methods(dirpath=None)#
A dictionary for method metadata. File data is saved in
methods.json
.- backup()#
Write a backup version of the data to the
backups
directory.
- clear() None. Remove all items from D. #
- deserialize()#
Load the serialized data. Can be replaced with other serialization formats.
- flush()#
Serialize the current data to disk.
- get(k[, d]) D[k] if k in D, else d. d defaults to None. #
- items() a set-like object providing a view on D's items #
- keys() a set-like object providing a view on D's keys #
- property list#
List the keys of the dictionary. This is a property, and does not need to be called.
- load()#
Load the serialized data. Creates the file if not yet present.
- pack(data)#
Transform the dictionary to a list because JSON can’t handle lists as keys
- pop(k[, d]) v, remove specified key and return the corresponding value. #
If key is not found, d is returned if given, otherwise KeyError is raised.
- popitem() (k, v), remove and return some (key, value) pair #
as a 2-tuple; but raise KeyError if D is empty.
- random()#
Return a random key.
- serialize(filepath=None)#
Method to do the actual serialization. Can be replaced with other serialization formats.
- Args:
filepath (str, optional): Provide an alternate filepath (e.g. for backup).
- setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D #
- unpack(data)#
Transform data back to a dictionary
- update([E, ]**F) None. Update D from mapping/iterable E and F. #
If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
- values() an object providing a view on D's values #
normalizations#
- class bw2data.meta.NormalizationMeta(dirpath=None)#
A dictionary for normalization metadata. File data is saved in
methods.json
.
weightings#
- class bw2data.meta.WeightingMeta(dirpath=None)#
A dictionary for weighting metadata. File data is saved in
methods.json
.
Mappings#
mapping#
- class bw2data.meta.Mapping(dirpath=None)#
A dictionary that maps object ids, like
("Ecoinvent 2.2", 42)
, to integers. Needed because parameter arrays have integerrow
andcolumn
fields.File data is saved in
mapping.pickle
.This dictionary does not support setting items directly; instead, use the
add
method to add multiple keys.- add(keys)#
Add a set of keys. These keys can already be in the mapping; only new keys will be added.
- Args:
keys (list): The keys to add.
- delete(keys)#
Delete a set of keys.
- Args:
keys (list): The keys to delete.
geomapping#
- class bw2data.meta.GeoMapping(*args, **kwargs)#
A dictionary that maps location codes to integers. Needed because parameter arrays have integer
geo
fields.File data is stored in
geomapping.pickle
.This dictionary does not support setting items directly; instead, use the
add
method to add multiple keys.
Data stores#
DataStore#
- class bw2data.DataStore(name)#
Base class for all Brightway2 data stores. Subclasses should define:
metadata: A Serialized Dictionary instance, e.g.
databases
ormethods
. The custom is that each type of data store has a new metadata store, so the data storeFoo
would have a metadata storefoos
.validator: A data validator. Optional. See bw2data.validate.
- backup()#
Save a backup to
backups
folder.- Returns:
File path of backup.
- copy(name)#
Make a copy of this object with a new
name
.This method only changes the name, but not any of the data or metadata.
- Args:
name (object): Name of the new object.
- Returns:
The new object.
- deregister()#
Remove an object from the metadata store. Does not delete any files.
- property filename#
Remove filesystem-unsafe characters and perform unicode normalization on
self.name
usingutils.safe_filename()
.
- load()#
Load the intermediate data for this object.
- Returns:
The intermediate data.
- register(**kwargs)#
Register an object with the metadata store. Takes any number of keyword arguments.
- validate(data)#
Validate data. Must be called manually.
- write(data)#
Serialize intermediate data to disk.
- Args:
data (object): The data
Inventory data backends#
DatabaseChooser#
The function bw2data.database.DatabaseChooser()
will choose the correct Database
class depending on the database backend registered for the database in its metadata. bw2data
comes with two backends (see Databases can be stored in different ways).
The function bw2data.database.Database
is an alias for bw2data.database.DatabaseChooser()
, provided for backwards compatibility.
- bw2data.database.DatabaseChooser(name, backend=None)#
A method that returns a database class instance. The default database type is SingleFileDatabase. JSONDatabase stores each process dataset in indented JSON in a separate file. Database types are specified in databases[database_name][‘backend’].
New database types can be registered with the config object:
config.backends['backend type string'] = MyNewBackendClass
Warning
Registering new backends must be done each time you start the Python interpreter.
To test whether an object is a database subclass, do:
from bw2data.backends import LCIBackend isinstance(my_database, LCIBackend)
Switching backends#
- bw2data.backends.utils.convert_backend(database_name, backend)#
Convert a Database to another backend.
bw2data currently supports the default and json backends.
- Args:
database_name (unicode): Name of database.
backend (unicode): Type of database. backend should be recoginized by DatabaseChooser.
Returns False if the old and new backend are the same. Otherwise returns an instance of the new Database object.
Custom database backends#
New database backends should inherit from bw2data.backends.base.LCIBackend
:
- class bw2data.backends.base.LCIBackend(name)#
A base class for LCI backends.
Subclasses must support at least the following calls:
load()
write(data)
In addition, they should specify their backend with the
backend
attribute (a unicode string).LCIBackend
provides the following, which should not need to be modified:rename
copy
find_dependents
random
process
For new classes to be recognized by the
DatabaseChooser
, they need to be registered with theconfig
object, e.g.:config.backends['backend type string'] = BackendClass
Instantiation does not load any data. If this database is not yet registered in the metadata store, a warning is written to
stdout
.The data schema for databases in voluptuous is:
exchange = { Required("input"): valid_tuple, Required("type"): basestring, } exchange.update(uncertainty_dict) lci_dataset = { Optional("categories"): Any(list, tuple), Optional("location"): object, Optional("unit"): basestring, Optional("name"): basestring, Optional("type"): basestring, Optional("exchanges"): [exchange] } db_validator = Schema({valid_tuple: lci_dataset}, extra=True)
- where:
valid_tuple
is a dataset identifier, like("ecoinvent", "super strong steel")
uncertainty_fields
are fields from an uncertainty dictionary.
Processing a Database actually produces two parameter arrays: one for the exchanges, which make up the technosphere and biosphere matrices, and a geomapping array which links activities to locations.
- Args:
name (unicode string): Name of the database to manage.
- copy(name)#
Make a copy of the database.
Internal links within the database will be updated to match the new database name, i.e.
("old name", "some id")
will be converted to("new name", "some id")
for all exchanges.- Args:
name (str): Name of the new database. Must not already exist.
- delete(**kwargs)#
Delete data from this instance. For the base class, only clears cached data.
- property filename#
Remove filesystem-unsafe characters and perform unicode normalization on
self.name
usingutils.safe_filename()
.
- find_dependents(data=None, ignore=None)#
Get sorted list of direct dependent databases (databases linked from exchanges).
- Args:
data (dict, optional): Inventory data
ignore (list): List of database names to ignore
- Returns:
List of database names
- find_graph_dependents()#
Recursively get list of all dependent databases.
- Returns:
A set of database names
- load(*args, **kwargs)#
Load the intermediate data for this database.
If
load()
does not return a dictionary, then the returned object must have at least the following dictionary-like methods:__iter__
__contains__
__getitem__
__setitem__
__delitem__
__len__
keys()
values()
items()
items()
However, this method must support the keyword argument
as_dict
, and.load(as_dict=True)
must return a normal dictionary with all Database data. This is necessary for JSON serialization.It is recommended to subclass
collections.{abc.}MutableMapping
(seeSynchronousJSONDict
for an example of data loaded on demand).
- process(*args, **kwargs)#
Process inventory documents.
Creates both a parameter array for exchanges, and a geomapping parameter array linking inventory activities to locations.
If the uncertainty type is no uncertainty, undefined, or not specified, then the ‘amount’ value is used for ‘loc’ as well. This is needed for the random number generator.
- Args:
version (int, optional): The version of the database to process
Doesn’t return anything, but writes two files to disk.
- query(*queries)#
Search through the database.
- random()#
Return a random activity key.
Returns a random activity key, or
None
(and issues a warning) if the current database is empty.
- register(**kwargs)#
Register a database with the metadata store.
Databases must be registered before data can be written.
- Writing data automatically sets the following metadata:
depends: Names of the databases that this database references, e.g. “biosphere”
number: Number of processes in this database.
- Args:
format (str, optional): Format that the database was converted from, e.g. “Ecospold”
- relabel_data(data, new_name)#
Relabel database keys and exchanges.
In a database which internally refer to the same database, update to new database name
new_name
.Needed to copy a database completely or cut out a section of a database.
For example:
data = { ("old and boring", 1): {"exchanges": [ {"input": ("old and boring", 42), "amount": 1.0}, ] }, ("old and boring", 2): {"exchanges": [ {"input": ("old and boring", 1), "amount": 4.0} ] } } print(relabel_database(data, "shiny new")) >> { ("shiny new", 1): {"exchanges": [ {"input": ("old and boring", 42), "amount": 1.0}, ] }, ("shiny new", 2): {"exchanges": [ {"input": ("shiny new", 1), "amount": 4.0} ] } }
In the example, the exchange to
("old and boring", 42)
does not change, as this is not part of the updated data.- Args:
data (dict): The data to modify
new_name (str): The name of the modified database
- Returns:
The modified data
- rename(name)#
Rename a database. Modifies exchanges to link to new name. Deregisters old database.
- Args:
name (str): New name.
- Returns:
New
Database
object.
- write(data)#
Serialize data to disk.
data
must be a dictionary of the form:{ ('database name', 'dataset code'): {dataset} }
Default backend - databases stored in a SQLite database#
This backend is a hybrid between SQLite and a document database. See also Storing Python objects in a SQLite3 database is silly! Why not use X document database?
The LCI data is stored in two tables, ActivityDataset and ExchangeDataset. Interactions with the database are mostly done using the Peewee ORM, although some raw SQL queries are used for performance reasons. The table have the following schemas:
CREATE TABLE "activitydataset" (
"id" INTEGER NOT NULL PRIMARY KEY,
"data" BLOB NOT NULL,
"code" TEXT NOT NULL,
"database" TEXT NOT NULL,
"location" TEXT,
"name" TEXT,
"product" TEXT,
"type" TEXT
)
CREATE TABLE "exchangedataset" (
"id" INTEGER NOT NULL PRIMARY KEY,
"data" BLOB NOT NULL,
"input_code" TEXT NOT NULL,
"input_database" TEXT NOT NULL,
"output_code" TEXT NOT NULL,
"output_database" TEXT NOT NULL,
"type" TEXT NOT NULL
)
As one of the fundamental principles of Brightway2 is to have a document-based backend, we incude most data in the column data, which is stored as a binary pickle. Serializing and deserializing is handled automatically by the Peewee interface. However, some attributes can be changed in the database. The following columns are canonical, i.e. they are included when constructing the Activity and Exchange objects:
ActivityDataset.database
ActivityDataset.code
ExchangeDataset.input_database
ExchangeDataset.input_code
ExchangeDataset.output_database
ExchangeDataset.output_code
All other columns are only used to querying and filtering datasets.
- class bw2data.backends.peewee.database.SQLiteBackend(*args, **kwargs)#
- delete(keep_params=False, warn=True)#
Delete all data from SQLite database and Whoosh index
- load(*args, **kwargs)#
Load the intermediate data for this database.
If
load()
does not return a dictionary, then the returned object must have at least the following dictionary-like methods:__iter__
__contains__
__getitem__
__setitem__
__delitem__
__len__
keys()
values()
items()
items()
However, this method must support the keyword argument
as_dict
, and.load(as_dict=True)
must return a normal dictionary with all Database data. This is necessary for JSON serialization.It is recommended to subclass
collections.{abc.}MutableMapping
(seeSynchronousJSONDict
for an example of data loaded on demand).
- process()#
Process inventory documents to NumPy structured arrays.
Use a raw SQLite3 cursor instead of Peewee for a ~2 times speed advantage.
- random(filters=True, true_random=False)#
True random requires loading and sorting data in SQLite, and can be resource-intensive.
- search(string, **kwargs)#
Search this database for
string
.The searcher include the following fields:
name
comment
categories
location
reference product
string
can include wild cards, e.g."trans*"
.By default, the
name
field is given the most weight. The full weighting set is called theboost
dictionary, and the default weights are:{ "name": 5, "comment": 1, "product": 3, "categories": 2, "location": 3 }
Optional keyword arguments:
limit
: Number of results to return.boosts
: Dictionary of field names and numeric boosts - see default boost values above. New values must be in the same format, but with different weights.filter
: Dictionary of criteria that search results must meet, e.g.{'categories': 'air'}
. Keys must be one of the above fields.mask
: Dictionary of criteria that exclude search results. Same format asfilter
.facet
: Field to facet results. Must be one ofname
,product
,categories
,location
, ordatabase
.proxy
: ReturnActivity
proxies instead of raw Whoosh documents. Default isTrue
.
Returns a list of
Activity
datasets.
- write(data, process=True)#
Write
data
to database.data
must be a dictionary of the form:{ ('database name', 'dataset code'): {dataset} }
Writing a database will first deletes all existing data.
Single file - each database in a single file#
- class bw2data.backends.single_file.database.SingleFileDatabase(name)#
A data store for LCI databases where each database is stored as a
pickle
file.Databases are automatically versioned. See below for reversion, etc. methods
- Args:
name (str): Name of the database to manage.
- property filename#
Remove filesystem-unsafe characters and perform unicode normalization on
self.name
usingutils.safe_filename()
.
- filename_for_version(version=None)#
Filename for given version; Default is current version.
- Returns:
Filename (not path)
- get(code)#
Get Activity proxy for this dataset
- load(version=None, **kwargs)#
Load the intermediate data for this database.
Can also load previous versions of this database’s intermediate data.
- Args:
version (int): Version of the database to load. Default
version
is the latest version.
- Returns:
The intermediate data, a dictionary.
- make_latest_version()#
Make the current version the latest version.
Requires loading data because a new intermediate data file is created.
- register(**kwargs)#
Register a database with the metadata store.
Databases must be registered before data can be written.
- revert(version)#
Return data to a previous state.
Warning
Reverting can lead to data loss, e.g. if you revert from version 3 to version 1, and then save your database, you will overwrite version 2. Use
make_latest_version()
before saving, which will set the current version to 4.- Args:
version (int): Number of the version to revert to.
- property version#
The current version number (integer) of this database.
- Returns:
Version number
- versions()#
Get a list of available versions of this database.
- Returns:
List of (version, datetime created) tuples.
- write(data, process=True)#
Serialize data to disk.
- Args:
data (dict): Inventory data
Version-control friendly - each database is a JSON file#
- class bw2data.backends.json.database.JSONDatabase(name)#
A data store for LCI databases. Stores each dataset in a separate file, serialized to JSON.
Instead of loading all the data at once,
.load()
creates aSynchronousJSONDict
, which loads values on demand.Use this backend by setting
"backend":"json"
in the database metadata. This is done automatically if you call.register()
from this class.- get(code)#
Get Activity proxy for this dataset
- load(as_dict=False, *args, **kwargs)#
Instantiate
SynchronousJSONDict
for this database.
- register(**kwargs)#
Register a database with the metadata store, using the correct value for
backend
, and creates database directory.
- write(data, process=True)#
Serialize data to disk. Most of the time, this data has already been saved to disk, so this is a no-op. The only exception is if
data
is a new database dictionary.Normalizes units when found.
- Args:
data (dict): Inventory data
- class bw2data.backends.json.sync_json_dict.SynchronousJSONDict(dirpath, dirname)#
A dictionary which stores each value as a separate file on disk. Values are loaded asynchronously (i.e. only as needed), but saved synchronously (i.e. immediately).
Dictionary keys are strings, and do not correspond with filenames. The utility function safe_filename is used to translate keys into allowable filenames, and a separate mapping dictionary is kept to map dictionary keys to filenames.
Retrieving a key returns a
frozendict
, which can’t be modified. This is to make sure that all changes get synced to disk. To change a dataset you must replace it completely, i.e. this won’t work (it will raise anAttributeError
):my_sync_dict['foo']['bar'] = 'baz'
Instead, you must do:
my_sync_dict['foo'] = {'bar': 'baz'}
After which the ‘foo’ file would be updated.
- filepath(key)#
Use
bw2data.utils.safe_filename()
to get filename for keykey
.
- from_json(data)#
Change exchange inputs from lists to tuples (as there is no distinction in JSON, but Python only allows tuples as dictionary keys).
- keys() a set-like object providing a view on D's keys #
- class bw2data.backends.json.sync_json_dict.frozendict(*args, **kw)#
A dictionary that can be created but not modified.
From http://code.activestate.com/recipes/414283-frozen-dictionaries/
- property clear#
- property pop#
If the key is not found, return the default if given; otherwise, raise a KeyError.
- property popitem#
Remove and return a (key, value) pair as a 2-tuple.
Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.
- property setdefault#
Insert key with a value of default if key is not in the dictionary.
Return the value for key if key is in the dictionary, else default.
- property update#
If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
Impact Assessment data stores#
ImpactAssessmentDataStore#
- bw2data.ia_data_store.abbreviate(names, length=8)#
Take a tuple or list, and construct a string, doing the following:
First, apply
utils.safe_filename()
to each element innames
.- Next, take the following, in order:
The first word of the first element in names, lower-cased, where word is defined as everything up to the first empty space character.
Join the rest of the first element (i.e. after the first word) with all other elements. Use the empty space character to join.
In this long string separated by spaces, take the lowercase first character of each word. Add the first word to this new string.
Finally, add a dash, and then the MD5 hash of the entire identifier, where each element is joined by a dash character.
('ReCiPe Endpoint (E,A)', 'human health', 'ionising radiation')
becomes'recipee(hhir-70eeef20a20deb6347ad428e3f6c5f3c'
.The MD5 hash is needed because taking the first characters doesn’t guarantee unique strings.
- class bw2data.ia_data_store.ImpactAssessmentDataStore(name)#
A subclass of
DataStore
for impact assessment methods.IA objects are hierarchically structured, and their identifier uses this structure, like
('ecological scarcity 2006', 'total', 'natural resources')
. The identifier must be atuple
, i.e.()
, not alist
, i.e.[]
. The identifier should only contain unicode strings, and can be of any length >= 1.Because impact assessment methods are identified by a tuple of strings, e.g.
('ReCiPe Endpoint (E,A)', 'human health', 'ionising radiation')
, we need to transform this identifier before it can be used e.g. as a filename. We do this using theabbreviate()
function, which returns a single unicode string.- Args:
name (tuple): Name of the IA object to manage. Must be a tuple of unicode strings.
- copy(name=None)#
Make a copy of the method, including its CFs and metadata.
If
name
is not provided, add “Copy of” to the last element of the original name, e.g.("foo", "bar")
becomes("foo", "Copy of bar")
- Args:
name (tuple, optional): Name of the new method.
- Returns:
The new object.
- property filename#
Remove filesystem-unsafe characters and perform unicode normalization on
self.name
usingutils.safe_filename()
.
- get_abbreviation()#
Retrieve the abbreviation of the method identifier from the metadata store. See class documentation.
- register(**kwargs)#
Register an object with the metadata store.
The metadata key
abbreviation
is set automatically.Objects must be registered before data can be written. If this object is not yet registered in the metadata store, a warning is written to stdout.
Takes any number of keyword arguments.
Method#
- class bw2data.method.Method(name)#
A manager for an impact assessment method. This class can register or deregister methods, write intermediate data, process data to parameter arrays, validate, and copy methods.
The Method class never holds intermediate data, but it can load or write intermediate data. The only attribute is name, which is the name of the method being managed.
Instantiation does not load any data. If this method is not yet registered in the metadata store, a warning is written to
stdout
.Methods are hierarchally structured, and this structure is preserved in the method name. It is a tuple of strings, like
('ecological scarcity 2006', 'total', 'natural resources')
.The data schema for IA methods is:
Schema([Any( [valid_tuple, maybe_uncertainty], # site-generic [valid_tuple, maybe_uncertainty, object] # regionalized )])
- where:
valid_tuple (tuple): A dataset identifier, like
("biosphere", "CO2")
.maybe_uncertainty (uncertainty dict or number): Either a number or an uncertainty dictionary.
object (object, optional) is a location identifier, used only for regionalized LCIA.
- Args:
name (tuple): Name of impact assessment method to manage.
- add_mappings(data)#
Add objects to
mapping
orgeomapping
, if necessary.- Args:
data (object): The data
- process_data(row)#
Translate data into correct order
- write(data, process=True)#
Serialize intermediate data to disk.
Sets the metadata key
num_cfs
automatically.
Normalization#
- class bw2data.weighting_normalization.Normalization(name)#
LCIA normalization data - used to transform meaningful units, like mass or damage, into “person-equivalents” or some such thing.
The data schema for IA normalization is:
Schema([ [valid_tuple, maybe_uncertainty] ])
- where:
valid_tuple
is a dataset identifier, like("biosphere", "CO2")
maybe_uncertainty
is either a number or an uncertainty dictionary
- add_mappings(data)#
Add each normalization flow (should be biosphere flows) to global mapping
- process_data(row)#
Return values that match
dtype_fields
, as well as number or uncertainty dictionary
Weighting#
- class bw2data.weighting_normalization.Weighting(name)#
LCIA weighting data - used to combine or compare different impact categories.
The data schema for weighting is a one-element list:
Schema(All( [uncertainty_dict], Length(min=1, max=1) ))
- process_data(row)#
Return an empty tuple (as
dtype_fields
is empty), and the weighting uncertainty dictionary.
- write(data)#
Because of DataStore assumptions, need a one-element list
User preferences#
- class bw2data.meta.Preferences(*args, **kwargs)#
A dictionary of project-specific preferences.
Parameters#
- class bw2data.parameters.ParameterManager#
- add_exchanges_to_group(group, activity)#
Add exchanges with formulas from
activity
togroup
.Every exchange with a formula field will have its original amount value stored as original_amount. This original value can be restored when parameterization is removed from the activity with remove_from_group.
- add_to_group(group, activity)#
Add activity to group.
Creates
group
if needed.Will delete any existing
ActivityParameter
for this activity.Deletes parameters key from Activity.
- new_activity_parameters(data, group, overwrite=True)#
Efficiently and correctly enter multiple parameters. Deletes all existing activity parameters for this group.
Will overwrite existing parameters in the same group with the same name, unless
overwrite
is false, in which case aValueError
is raised.Input parameters must refer to a single, existing database.
group
is the group name; will be autocreated if necessary.data
should be a list of dictionaries:[{ 'name': name of variable (unique), 'database': activity database, 'code': activity code, 'amount': numeric value of variable (optional), 'formula': formula in Python as string (optional), optional keys like uncertainty, etc. (no limitations) }]
- new_database_parameters(data, database, overwrite=True)#
Efficiently and correctly enter multiple parameters. Deletes all existing database parameters for this database.
Will overwrite existing database parameters with the same name, unless
overwrite
is false, in which case aValueError
is raised.database
should be an existing database.data
should be a list of dictionaries:[{ 'name': name of variable (unique), 'amount': numeric value of variable (optional), 'formula': formula in Python as string (optional), optional keys like uncertainty, etc. (no limitations) }]
- new_project_parameters(data, overwrite=True)#
Efficiently and correctly enter multiple parameters.
Will overwrite existing project parameters with the same name, unless
overwrite
is false, in which case aValueError
is raised.data
should be a list of dictionaries:[{ 'name': name of variable (unique), 'amount': numeric value of variable (optional), 'formula': formula in Python as string (optional), optional keys like uncertainty, etc. (no limitations) }]
- recalculate()#
Recalculate all expired project, database, and activity parameters, as well as exchanges.
- remove_exchanges_from_group(group, activity, restore_original=True)#
Takes a group and activity and removes all
ParameterizedExchange
objects from the group.The
restore_original
parameter determines if the original amount values will be restored to those exchanges where a formula was used to alter the amount.
- remove_from_group(group, activity, restore_amounts=True)#
Remove activity from group.
Will delete any existing
ActivityParameter
andParameterizedExchange
for this activity.Restores parameters key to this Activity. By default, restores amount value of each parameterized exchange of the Activity to the original value. This can be avoided by using the
restore_amounts
parameter.
- rename_activity_parameter(parameter, new_name, update_dependencies=False)#
Given a parameter and a new name, safely update the parameter.
Will raise a TypeError if the given parameter is of the incorrect type. Will raise a ValueError if other parameters depend on the given one and
update_dependencies
is False.
- rename_database_parameter(parameter, new_name, update_dependencies=False)#
Given a parameter and a new name, safely update the parameter.
Will raise a TypeError if the given parameter is of the incorrect type. Will raise a ValueError if other parameters depend on the given one and
update_dependencies
is False.
- rename_project_parameter(parameter, new_name, update_dependencies=False)#
Given a parameter and a new name, safely update the parameter.
Will raise a TypeError if the given parameter is of the incorrect type. Will raise a ValueError if other parameters depend on the given one and
update_dependencies
is False.
- class bw2data.parameters.Group(*args, **kwargs)#
- DoesNotExist#
alias of
GroupDoesNotExist
- expire()#
Set
fresh
toFalse
- freshen()#
Set
fresh
toTrue
- save(*args, **kwargs)#
Save this model instance. Will remove ‘project’ and database names from
order
.
- class bw2data.parameters.ProjectParameter(*args, **kwargs)#
Parameter set for a project. Group name is ‘project’.
Columns:
name: str, unique
formula: str, optional
amount: float, optional
data: object, optional. Used for any other metadata.
Note that there is no magic for reading and writing to
data
(unlikeActivity
objects) - it must be used directly.- DoesNotExist#
alias of
ProjectParameterDoesNotExist
- static dependency_chain()#
Determine if
`ProjectParameter
parameters have dependencies within the group.Returns:
[ { 'kind': 'project', 'group': 'project', 'names': set of variables names } ]
- property dict#
Parameter data as a standardized dictionary
- static expired()#
Return boolean - is this group expired?
- is_deletable()#
Perform a test to see if the current parameter can be deleted.
- static load(group=None)#
Return dictionary of parameter data with names as keys and
.dict()
as values.
- static recalculate(ignored=None)#
Recalculate all parameters.
ignored
included for API compatibility with otherrecalculate
methods - it will really be ignored.
- static static(ignored='project', only=None)#
Get dictionary of
{name: amount}
for all project parameters.only
restricts returned names to ones found inonly
.ignored
included for API compatibility with otherrecalculate
methods.
- classmethod update_formula_parameter_name(old, new)#
Performs an update of the formula of relevant parameters.
NOTE: Make sure to wrap this in an .atomic() statement!
- class bw2data.parameters.DatabaseParameter(*args, **kwargs)#
Parameter set for a database. Group name is the name of the database.
Columns:
database: str
name: str, unique within a database
formula: str, optional
amount: float, optional
data: object, optional. Used for any other metadata.
Note that there is no magic for reading and writing to
data
(unlikeActivity
objects) - it must be used directly.- DoesNotExist#
alias of
DatabaseParameterDoesNotExist
- static dependency_chain(group, include_self=False)#
Find where each missing variable is defined in dependency chain.
If
include_self
is True will include parameters within the group as possible dependenciesReturns:
[ { 'kind': one of 'project', 'database', 'activity', 'group': group name, 'names': set of variables names } ]
- property dict#
Parameter data as a standardized dictionary
- static expired(database)#
Return boolean - is this group expired?
- is_deletable()#
Perform a test to see if the current parameter can be deleted.
- static is_dependent_on(name)#
Test if any database parameters are dependent on the given project parameter name.
- static load(database)#
Return dictionary of parameter data with names as keys and
.dict()
as values.
- static recalculate(database)#
Recalculate all database parameters for
database
, if expired.
- save(*args, **kwargs)#
Save this model instance
- static static(database, only=None)#
Return dictionary of {name: amount} for database group.
- classmethod update_formula_database_parameter_name(old, new)#
Performs an update of the formula of relevant parameters.
This method specifically targets database parameters used in database formulas
- classmethod update_formula_project_parameter_name(old, new)#
Performs an update of the formula of relevant parameters.
This method specifically targets project parameters used in database formulas
- class bw2data.parameters.ActivityParameter(*args, **kwargs)#
Parameter set for a group of activities.
Columns:
group: str
database: str
code: str. Code and database define the linked activity for this parameter.
name: str, unique within a group
formula: str, optional
amount: float, optional
data: object, optional. Used for any other metadata.
Activities can only have parameters in one group. Group names cannot be ‘project’ or the name of any existing database.
Activity parameter groups can depend on other activity parameter groups, so that a formula in group “a” can depend on a variable in group “b”. This dependency information is stored in
Group.order
- in our small example, we could define the following:a = Group.get(name="a") a.order = ["b", "c"] a.save()
In this case, a variable not found in “a” would be searched for in “b” and then “c”, in that order. Database and then project parameters are also implicitly included at the end of
Group.order
.Note that there is no magic for reading and writing to
data
(unlikeActivity
objects) - it must be used directly.- DoesNotExist#
alias of
ActivityParameterDoesNotExist
- static dependency_chain(group, include_self=False)#
Find where each missing variable is defined in dependency chain.
Will also load in all parameters needed to resolve the
ParameterizedExchanges
for this group.If
include_self
is True will include parameters within the group as possible dependenciesReturns:
[ { 'kind': one of 'project', 'database', 'activity', 'group': group name, 'names': set of variables names } ]
- property dict#
Parameter data as a standardized dictionary
- static expired(group)#
Return boolean - is this group expired?
- is_deletable()#
Perform a test to see if the current parameter can be deleted.
- static is_dependency_within_group(name, group, include_order=False)#
Determine if the given parameter name is a dependency within the given activity group.
The optional
include_order
parameter will include dependencies from groups found in the theGroup
.`order` field.
- static is_dependent_on(name, group)#
Test if any activity parameters are dependent on the given parameter name from the given group.
- static load(group)#
Return dictionary of parameter data with names as keys and
.dict()
as values.
- static recalculate(group)#
Recalculate all values for activity parameters in this group, and update their underlying Activity and Exchange values.
- static recalculate_exchanges(group)#
Recalculate formulas for all parameterized exchanges in group
group
.
- save(*args, **kwargs)#
Save this model instance
- static static(group, only=None, full=False)#
Get dictionary of
{name: amount}
for parameters defined ingroup
.only
restricts returned names to ones found inonly
.full
returns all names, including those found in the dependency chain.
- classmethod update_formula_activity_parameter_name(old, new, include_order=False)#
Performs an update of the formula of relevant parameters.
This method specifically targets activity parameters used in activity formulas
- classmethod update_formula_database_parameter_name(old, new)#
Performs an update of the formula of relevant parameters.
This method specifically targets database parameters used in activity formulas
- classmethod update_formula_project_parameter_name(old, new)#
Performs an update of the formula of relevant parameters.
This method specifically targets project parameters used in activity formulas
Utilities#
- utils.combine_methods(*ms)#
Combine LCIA methods by adding duplicate characterization factors.
- Args:
ms (one or more method id tuples): Any number of method ids, e.g.
("my method", "wow"), ("another method", "wheee")
.
- Returns:
The new Method instance.
- utils.download_file(directory='downloads', url=None)#
Download a file and write it to disk in
downloads
directory.If
url
is None, uses the Brightway2 data base URL.url
should everything up to the filename, such thaturl
+filename
is the valid complete URL to download from.Streams download to reduce memory usage.
- Args:
filename (str): The filename to download.
directory (str, optional): Directory to save the file. Created if it doesn’t already exist.
url (str, optional): URL where the file is located, if not the default Brightway data URL.
- Returns:
The path of the created file.
- utils.natural_sort()#
Sort the given list in the way that humans expect, e.g. 9 before 10.
- utils.random_string()#
Generate a random string of letters and numbers.
- Args:
length (int): Length of string, default is 8
- Returns:
A string (not unicode)
- utils.recursive_str_to_unicode(encoding='utf8')#
Convert the strings inside a (possibly nested) python data structure to unicode strings using encoding.
- utils.uncertainify(distribution=None, bounds_factor=0.1, sd_factor=0.1)#
Add some rough uncertainty to exchanges.
Warning
This function only changes exchanges with no uncertainty type or uncertainty type
UndefinedUncertainty
, and does not change production exchanges!Can only apply normal or uniform uncertainty distributions; default is uniform. Distribution, if specified, must be a
stats_array
uncertainty object.data
is a LCI data dictionary.If using the normal distribution:
sd_factor
will be multiplied by the mean to calculate the standard deviation.If no bounds are desired, set
bounds_factor
toNone
.Otherwise, the bounds will be
[(1 - bounds_factor) * mean, (1 + bounds_factor) * mean]
.
If using the uniform distribution, then the bounds are
[(1 - bounds_factor) * mean, (1 + bounds_factor) * mean]
.Returns the modified data.