AERSHED Platform

Appearance

Infrastructure import ​

The Infrastructure Data Import API provides a streamlined way to import infrastructure data into the system.

Users interact with the API by providing JSON files that define the operations and data to be imported. The API supports various operations for creating and updating infrastructure sites and equipment.

Supported Operations ​

The API supports the following operations:

  • CREATE_SITES: Create new infrastructure sites.
  • UPDATE_SITES: Update existing infrastructure sites.
  • CREATE_EQUIPMENT: Create new equipment entries.
  • UPDATE_EQUIPMENT: Update existing equipment entries.

These operations are defined in the import_operations module and are automatically recognized by the importer.

JSON File Structure ​

To use the API, users must provide JSON files with a specific structure. Each JSON file should contain a list of operations, where each operation includes the following keys:

  • operation: The type of operation to perform. Must be one of the supported operations listed above.
  • params: A dictionary of parameters specific to the operation.
  • data: A list of data items to be processed by the operation.

Example JSON Structure

json
[
    {
        "operation": "CREATE_SITES",
        "params": {
            "cross_operator_match_distance_m": 100
        },
        "data": [
            {
                "owner": "INFRA_01",
                "site_name": "Site A",
                "geometry": "{\"type\": \"Point\",\"coordinates\": [-106.8436308791011, 34.783062571462494]}",
								"equipment": [
									{ ... }
								],
								...
            },
            {
                "owner": "INFRA_01",
                "site_name": "Site B",
                ...
            }
        ]
    },
    {
        "operation": "CREATE_EQUIPMENT",
        "params": {
            "owner_name": "INFRA_01",
            "site_name": "Site C"
        },
        "data": [
            {
                "equipment_class": "Compressor",
                "equipment_sub_class": "Reciprocating",
                "manufacturer_make": "GE",
                "manufacturer_model": "XYZ-123",
                "operator_unique_id": "COMP-001",
                "equipment_type": "EQUIPMENT",
                "geometry": "{\"type\": \"Point\",\"coordinates\": [-106.84104739463146, 34.78220541516408]}",
                ...
            }
        ]
    }
]

How to import data ​

Use the helper scripts provided by the data-pipelines repository.

  1. Ensure you have that repository set up and with a valid API key.
  2. Prepare a folder with one or more JSON files containing the operations you want to perform.
  3. Run the following command:
json
python -m importers.infrastructure.infrastructure --api_key API_KEY_HERE --environment ENVIRONMENT --data_path PATH_TO_FOLDER_WITH_JSONS

A note on error handling ​

The import process follows a all-or-nothing rule - it’ll try to process the operations sequentially, and if anything fails, it rolls back the entire process.

There’s not a good way to read the errors from the importers yet, but the messages are all being stored on the database.

  • Emission Import — import emission data from satellite and aerial providers
  • Matching — how imported infrastructure triggers emission matching
  • Sensor Data — sensor configuration for sites (the sensor_config field)
  • Site operations — detailed reference for CREATE_SITES, UPDATE_SITES, TRANSFER_SITES, MERGE_SITES, SPLIT_SITES
  • Equipment operations — detailed reference for CREATE_EQUIPMENT, UPDATE_EQUIPMENT