Pulse Measurement Data Flow

This document explains how pulse measurement data flows through the Atria installer application, from frontend user input through to the LoRaWAN device tables.

Overview

Pulse meters are devices that measure consumption (water, gas, electricity) by counting pulses. Each pulse represents a certain amount of the measured resource. The system supports up to 3 pulse counters per device.

Key fields per pulse counter:

Field	Purpose	Example Values

Field	Purpose	Example Values
`deviceType`	Measurement category	`PWM` (water), `PGM` (gas), `PEM` (electricity)
`scalar`	Conversion factor (pulses → units)	`0.001`, `0.264172`
`units`	Output unit	`gal`, `m3`, `ccf`, `kW`

Frontend: User Input & State Management

State Initialization

The pulse data journey begins in InstallerDeviceFormPage.jsx where the state is initialized:

const [pulseData, setPulseData] = useState({
  pulse1: { deviceType: '', scalar: '', units: '' },
  pulse2: { deviceType: '', scalar: '', units: '' },
  pulse3: { deviceType: '', scalar: '', units: '' }
});

Device Type Options

The PulseCounterSection.jsx component provides device type options from the taxonomy:

PWM - Pulse Water Meter
PGM - Pulse Gas Meter
PEM - Pulse Electricity Meter

Unit Filtering

Units are filtered based on the selected device type:

Device Type	Available Units

Device Type	Available Units
PWM (water)	`gal`, `liters`, `m3`
PGM (gas)	`m3`, `ccf`, `therms`
PEM (electricity)	`kW`

Validation

Each pulse counter must be either:

Completely empty (all three fields blank), OR
Completely filled (all three fields populated)

Partial data is rejected with validation errors.

Frontend: Form Submission

When the user submits the form, installerDeviceService.js packages the data:

// Add pulse data if present
if (installationData.pulseData) {
    formData.append("pulse_data", JSON.stringify(installationData.pulseData));
}

// Add pulse counters capability flag
if (installationData.hasPulseCounters !== undefined) {
    formData.append("has_pulse_counters", installationData.hasPulseCounters.toString());
}

Example data sent to backend:

{
  "pulse_data": "{\"pulse1\":{\"deviceType\":\"PWM\",\"scalar\":\"1\",\"units\":\"liters\"},\"pulse2\":{\"deviceType\":\"\",\"scalar\":\"\",\"units\":\"\"},\"pulse3\":{\"deviceType\":\"\",\"scalar\":\"\",\"units\":\"\"}}",
  "has_pulse_counters": "true"
}

Backend: Installation Lambda Processing

The installation_submission_lambda.py processes pulse data in several stages:

1. Taxonomy Code Conversion

Device types are standardized to taxonomy codes:

DEVICE_TYPE_TAXONOMY = {
    'pulse water meter': 'PWM',
    'pulse gas meter': 'PGM',
    'pulse electricity meter': 'PEM',
    'water meter': 'WM',
    'weather station': 'WS',
    'air quality': 'AQ',
    'gateway': 'GW',
    # ... more mappings
}

The convert_device_type_to_taxonomy_code() function handles:

Already-valid codes (passes through)
Human-readable names (converts to code)
Partial matches (best-effort conversion)
Unknown types (defaults to GPIO)

2. Unit Standardization

Values are converted to standard units based on device type:

def standardize_pulse_value(device_type, value, unit):
    if 'water' in device_type_lower or device_type_lower == 'pwm':
        standardized_value = convert_to_gallons(value, unit)
        return standardized_value, 'gal'
    elif 'gas' in device_type_lower or device_type_lower == 'pgm':
        standardized_value = convert_to_m3(value, unit)
        return standardized_value, 'm3'
    elif 'electricity' in device_type_lower or device_type_lower == 'pem':
        return convert_to_kw(value, unit), 'kW'

Conversion formulas:

From	To	Formula

From	To	Formula
liters	gallons	`value × 0.264172`
m³	gallons	`value × 264.172`
ccf	m³	`value × 2.8316846592`
therms	m³	`value × 29.3`

3. DynamoDB Record Creation

The lambda saves both raw and standardized values:

# Store raw user input (for reference/debugging)
device_record[f'PulseScaler{i}Raw'] = f"{scalar} {units}"  # e.g., "1 liters"

# Store standardized value (for migration lambda)
device_record[f'PulseScaler{i}'] = f"{standardized_value} {standardized_unit}"  # e.g., "0.264172 gal"

# Store device type taxonomy code
device_record[f'DeviceType{i}'] = taxonomy_code  # e.g., "PWM"

4. Validation Logic

# CRITICAL: If device has pulse counter capability, must have actual pulse data
if has_pulse_counters_capability and not has_active_pulse_counters:
    return build_response(400, {
        'error': 'Missing required pulse counter data',
        'message': 'Device is configured with pulse counter capability but no pulse counter data was provided.'
    })

Database Schema

InstallerAppDeviceDetails Table

Field	Type	Example Value	Description

Field	Type	Example Value	Description
`EntityId`	String	`a1b2c`	Device ID (5-char)
`SecondaryId`	String	`SITE1`	Site ID
`HasPulseCounters`	Boolean	`true`	Capability flag
`DeviceType`	String	`LWM`	Main device type
`DeviceType1`	String	`PWM`	Pulse 1 measurement type
`DeviceType2`	String	`PGM`	Pulse 2 measurement type
`DeviceType3`	String	``	Pulse 3 (empty if unused)
`PulseScaler1`	String	`0.264172 gal`	Standardized conversion
`PulseScaler2`	String	`2.83168 m3`	Standardized conversion
`PulseScaler3`	String	``	Empty if unused
`PulseScaler1Raw`	String	`1 liters`	Original user input
`PulseScaler2Raw`	String	`1 ccf`	Original user input
`PulseScaler3Raw`	String	``	Empty if unused

Migration Lambda: Moving to LoRaWAN Tables

The migration_lambda.py copies data from InstallerAppDeviceDetails to the LoRaWAN devices table for actual sensor operation.

Virtual Child Device Creation

For pulse-capable sensors, the migration creates virtual child devices:

has_pulse_counters = ddb_item.get("HasPulseCounters", False)

if has_pulse_counters:
    for i in range(1, 4):
        device_type = ddb_item.get(f"DeviceType{i}")
        child_dev_eui = f"{dev_eui}_P{i}"  # Creates virtual child devices!

        if device_type:
            # Parse "0.264172 gal" into value and unit
            pulse_scaler_str = ddb_item.get(f"PulseScaler{i}", "")
            parts = pulse_scaler_str.strip().split(" ", 1)
            pulse_scalar_value = parts[0]      # "0.264172"
            pulse_scalar_unit = parts[1]       # "gal"

            # Update LoRaWAN table with child device
            lorawan_devices_table.update_item(
                Key={"DevEui": child_dev_eui},
                UpdateExpression="SET DeviceType = :deviceType, PulseScalarValue = :val, PulseScalarUnit = :unit, ...",
                ExpressionAttributeValues={
                    ":deviceType": device_type,
                    ":val": pulse_scalar_value,
                    ":unit": pulse_scalar_unit,
                }
            )

Parent Device Handling

The parent device is marked as Invisible since actual data comes from child devices:

if has_pulse_counters:
    update_expression_parts.append("Invisible = :invisible")
    expression_attribute_values[':invisible'] = True
    device_type_to_set = "LWM"  # Legacy Water Meter for parent

Orphan Cleanup

When pulse counters are removed, orphaned child entries are deleted:

# Check for orphaned pulse entries and delete them
orphaned_keys_to_check = [{'DevEui': f"{dev_eui}_P{i}"} for i in range(1, 4)]
response = dynamodb.batch_get_item(...)
# Delete any that exist but shouldn't

Data Flow Diagram

┌──────────────────────────────────────────────────────────────────────────────┐
│  FRONTEND                                                                     │
│  ┌─────────────────────┐    ┌─────────────────────────────────────────┐      │
│  │  PulseCounterSection │    │  installerDeviceService.js              │      │
│  │  ─────────────────── │    │  ─────────────────────────────────────  │      │
│  │  deviceType: "PWM"   │───►│  formData.append("pulse_data",          │      │
│  │  scalar: "1"         │    │    JSON.stringify(pulseData))           │      │
│  │  units: "liters"     │    │  formData.append("has_pulse_counters",  │      │
│  └─────────────────────┘    │    "true")                              │      │
│                              └──────────────────┬──────────────────────┘      │
└─────────────────────────────────────────────────┼────────────────────────────┘
                                                  │ POST /installations
                                                  ▼
┌──────────────────────────────────────────────────────────────────────────────┐
│  INSTALLATION SUBMISSION LAMBDA                                               │
│  ┌────────────────────────────────────────────────────────────────────┐      │
│  │  1. Parse pulse_data JSON                                          │      │
│  │  2. Convert deviceType → taxonomy code (PWM stays PWM)             │      │
│  │  3. Standardize: convert_to_gallons(1, "liters") = 0.264172 gal   │      │
│  │  4. Save to DynamoDB:                                              │      │
│  │     • PulseScaler1Raw = "1 liters"                                 │      │
│  │     • PulseScaler1 = "0.264172 gal"                                │      │
│  │     • DeviceType1 = "PWM"                                          │      │
│  │     • HasPulseCounters = true                                      │      │
│  └────────────────────────────────────────────────────────────────────┘      │
└──────────────────────────────────────────────────┬───────────────────────────┘
                                                   │
                                                   ▼
┌──────────────────────────────────────────────────────────────────────────────┐
│  DynamoDB: InstallerAppDeviceDetails                                          │
│  ┌────────────────────────────────────────────────────────────────────┐      │
│  │  EntityId: "abc12"         DeviceType: "LWM"                       │      │
│  │  HasPulseCounters: true    DeviceType1: "PWM"                      │      │
│  │  PulseScaler1: "0.264172 gal"   PulseScaler1Raw: "1 liters"        │      │
│  └────────────────────────────────────────────────────────────────────┘      │
└──────────────────────────────────────────────────┬───────────────────────────┘
                                                   │ Migration Lambda (async)
                                                   ▼
┌──────────────────────────────────────────────────────────────────────────────┐
│  MIGRATION LAMBDA                                                             │
│  ┌────────────────────────────────────────────────────────────────────┐      │
│  │  For HasPulseCounters=true:                                        │      │
│  │  • Create abc12_P1 with DeviceType=PWM, PulseScalarValue=0.264172 │      │
│  │  • Create abc12_P2, abc12_P3 (if DeviceType2/3 exist)              │      │
│  │  • Mark parent abc12 as Invisible=true, DeviceType=LWM            │      │
│  └────────────────────────────────────────────────────────────────────┘      │
└──────────────────────────────────────────────────┬───────────────────────────┘
                                                   │
                                                   ▼
┌──────────────────────────────────────────────────────────────────────────────┐
│  DynamoDB: LoRaWAN Devices Table                                              │
│  ┌───────────────────────────────────────────────────────────────────┐       │
│  │  Parent Device:                                                    │       │
│  │  DevEui: "abc12...full"    DeviceType: "LWM"    Invisible: true   │       │
│  ├───────────────────────────────────────────────────────────────────┤       │
│  │  Child Device 1:                                                   │       │
│  │  DevEui: "abc12...full_P1"  DeviceType: "PWM"                      │       │
│  │  PulseScalarValue: "0.264172"  PulseScalarUnit: "gal"              │       │
│  └───────────────────────────────────────────────────────────────────┘       │
└──────────────────────────────────────────────────────────────────────────────┘

Unit Conversion Reference

Water (PWM) - Standard Unit: Gallons

Input Unit	Conversion Factor	Example

Input Unit	Conversion Factor	Example
gallons	1.0	1 gal → 1 gal
liters	0.264172	1 liter → 0.264172 gal
m³	264.172	1 m³ → 264.172 gal

Gas (PGM) - Standard Unit: Cubic Meters (m³)

Input Unit	Conversion Factor	Example

Input Unit	Conversion Factor	Example
m³	1.0	1 m³ → 1 m³
ccf	2.8316846592	1 ccf → 2.832 m³
therms	29.3	1 therm → 29.3 m³

Electricity (PEM) - Standard Unit: Kilowatts (kW)

Input Unit	Conversion Factor	Example

Input Unit	Conversion Factor	Example
kW	1.0	1 kW → 1 kW

Summary

User enters: deviceType=PWM, scalar=1, units=liters
Frontend sends: JSON-stringified pulseData + capability flag
Lambda processes: Converts 1 liter → 0.264172 gallons, saves both raw and standardized
Migration creates: Virtual child devices (_P1, _P2, _P3) with parsed scalar value/unit
Result: LoRaWAN system receives properly formatted conversion factors for each pulse counter

Related Files

Frontend
- frontend/src/pages/InstallerDeviceFormPage.jsx - Main form page
- frontend/src/components/installer/PulseCounterSection.jsx - Pulse counter UI
- frontend/src/services/installerDeviceService.js - API submission service
Backend
- backend/installer/installation_submission_lambda.py - Installation processing
- backend/installer/migration_lambda.py - LoRaWAN table migration

Pulse Measurement Data Flow

Pulse Measurement Data Flow

Table of Contents

Overview

Frontend: User Input & State Management

State Initialization

Device Type Options

Unit Filtering

Validation

Frontend: Form Submission

Backend: Installation Lambda Processing

1. Taxonomy Code Conversion

2. Unit Standardization

3. DynamoDB Record Creation

4. Validation Logic

Database Schema

InstallerAppDeviceDetails Table

Migration Lambda: Moving to LoRaWAN Tables

Virtual Child Device Creation

Parent Device Handling

Orphan Cleanup

Data Flow Diagram

Unit Conversion Reference

Water (PWM) - Standard Unit: Gallons

Gas (PGM) - Standard Unit: Cubic Meters (m³)

Electricity (PEM) - Standard Unit: Kilowatts (kW)

Summary

Related Files