RAPID Ingestion

RAPID ingestion is the Iron Horse intake path for waybill, railcar, and order context coming from the RAPID workflow. It is not just a raw import. The API server authenticates the source, stores current waybill records, preserves history when payloads change, and can create or represent BIRCH work orders from a model order when the RAPID data is safe enough to use.

What RAPID contributes

RAPID supplies operational/business context that complements physical yard signals such as AEI reads.

Typical RAPID payload data includes:

Waybill ID and waybill number
Source system
Equipment initial and number
Equipment type
Equipment owner
Origin and destination station/code
Route
Commodity and commodity code
Gross/tare/net weight
Hazmat flag
Shipper and consignee
Waybill and shipped dates
Estimated arrival
Current status, status location, and last update
Billing rate/currency/terms
Priority, load/empty, bill of lading, and payload version
Full raw payload for audit/replay/debugging

RAPID should be understood as the business/order signal. It says what the external workflow knows about a car, waybill, customer, route, commodity, and expected timing.

Current production endpoints

RAPID is mounted under the Iron Horse API server:

API server: https://api.ihrailsoftware.com
Route prefix: /api/rapid
Implementation: /home/ubuntu/api-server/src/rapid-routes.ts
Compiled route: /home/ubuntu/api-server/dist/rapid-routes.js
UI page: /RAPID and /public/rapid.html

Primary endpoints:

POST /api/rapid/ingest
POST /api/rapid/birch/model-order
GET  /api/rapid/waybills
GET  /api/rapid/waybills/:waybill_id

Do not document, paste, or commit RAPID API keys, Supabase service keys, database passwords, or raw credential URLs. The production code must load those from approved server-side environment/secret locations.

High-level flow

RAPID ingestion has two related but distinct jobs:

Waybill ingest stores and versions RAPID waybill data.
BIRCH model-order import creates or represents a BIRCH work order for a RAPID railcar using a known model work order.

Authentication model

Waybill ingest

POST /api/rapid/ingest requires an API key in:

x-api-key: <client key>

The server hashes the presented key with SHA-256 and looks it up in the RAPID api_clients table. The client must exist and be active.

The authenticated client identity is attached to the request as:

client_id
client_name

That client ID becomes part of the waybill record and dedupe key.

BIRCH model-order endpoint

POST /api/rapid/birch/model-order supports the normal RAPID API-client lookup and also supports a configured server-side list of pre-hashed API keys for the BIRCH order-import use case.

This keeps the BIRCH model-order import callable by trusted integrations without exposing secrets in frontend code.

Waybill ingest behavior

Endpoint:

POST /api/rapid/ingest

Expected body shape:

{
  "waybills": [
    {
      "waybill_id": "...",
      "source": "..."
    }
  ]
}

The payload may be nested or flat. The server supports both styles. For example, equipment values can come from either:

{
  "equipment": {
    "initial": "ABC",
    "number": "1234",
    "type": "TANK"
  }
}

or flat fields such as:

{
  "equipment_initial": "ABC",
  "equipment_number": "1234",
  "equipment_type": "TANK"
}

Required fields

Each waybill must have:

waybill_id
source

Rows missing either value are skipped.

Current-record model

The ingest endpoint stores waybills in Supabase using an is_current flag.

For each incoming waybill, the server checks for an existing current record matching:

waybill_id
source
client_id
is_current = true

Then it decides:

unchanged payload → skipped
changed payload   → old current row set to is_current=false, new row inserted
new payload       → inserted

This makes the table both a current-state store and a change-history store.

Dedupe behavior

If the incoming raw payload is identical to the current stored raw payload, the endpoint skips insertion. That prevents repeated sends from creating noisy duplicate history rows.

If the payload changed, the old current row is retained as history and the new payload becomes current.

Stored fields

The waybill record stores normalized columns for common UI/query needs and also stores the full raw payload.

Important columns include:

client_id
source
waybill_id
waybill_number
equipment_initial
equipment_number
equipment_type
equipment_owner
origin_station
origin_code
destination_station
destination_code
route
commodity
commodity_code
weight_gross
weight_tare
weight_net
hazmat
shipper_name
consignee_name
waybill_date
shipped_date
estimated_arrival
status_current
status_last_update
status_location
billing_rate
billing_currency
billing_terms
priority
load_empty
bill_of_lading
commodity_description
payload_version
received_at
raw_payload
is_current

RAPID viewer behavior

The RAPID UI is protected by Google SSO / app permission.

Routes:

/RAPID
/public/rapid.html

Viewer API endpoints:

GET /api/rapid/waybills
GET /api/rapid/waybills/:waybill_id

The list endpoint returns the latest current waybills, ordered by newest received time, limited to 500 records.

The detail endpoint returns:

current → current waybill row
history → older rows for the same waybill_id

This lets operators see what RAPID currently says and how that payload has changed over time.

BIRCH model-order import

Endpoint:

POST /api/rapid/birch/model-order

This endpoint is for creating or representing a BIRCH work order from RAPID railcar/order data.

It uses a known BIRCH model work order as a template:

Model work order: WO_05923

The model order supplies the default work-order structure, routing matrix tasks, job lines, job parts, optional billing rows, storage information, checklist report rows, and status/workupdate pattern.

Railcar identity

The endpoint accepts railcar identity from any of these payload fields:

railcar_id
car
rfid

The value is normalized by removing non-alphanumeric characters and uppercasing it.

BIRCH has a railcar ID length constraint, so the normalized ID must be 10 characters or fewer.

Existing active work order behavior

Before creating anything, the endpoint checks BIRCH for an active work order matching:

railcar_id = normalized railcar ID
shipped_date = 1900-01-01 00:00:00

If one exists, the endpoint does not create a duplicate work order. Instead it returns the existing work order summary with:

action: represented

If RAPID provided an estimated arrival date, the existing work order can be updated with that value.

If RAPID provided equipment type context, the railcar type can also be updated.

New BIRCH order behavior

If no active BIRCH work order exists for the railcar, the endpoint creates one by cloning the model order.

The process includes:

Start a BIRCH database transaction.
Confirm the model work order exists.
Normalize/validate the railcar ID.
Translate RAPID equipment type into a BIRCH railcar type.
Insert the railcar if needed by cloning model railcar defaults.
Allocate the next WO_##### work order number under lock.
Insert a new BIRCH workorder row from the model order.
Clone model job lines.
Clone model job parts.
Add a workupdate indicating creation by the selected user.
Clone routing matrix task assignments.
Clone supported optional rows.
Commit the transaction.
Return a summary.

If anything fails, the transaction rolls back.

Returned action values

The BIRCH model-order endpoint returns one of the major actions:

inserted     → new BIRCH work order created
represented  → existing active BIRCH work order found and represented

It also returns summary fields such as:

work_order
railcar_id
estimated_arrival_date
arrival_date
reason_to_come
routing_matrix
joblines
routing_tasks
status_id
railcar_action
railcar_type_translation

Equipment-type translation

RAPID equipment type/commodity text is translated to BIRCH railcar type IDs.

Current examples:

TANK / DOT 111 / DOT 117 → TANK
COVERED HOPPER / CVD HPR → COVERED HOPPER
HOPPER / HPR → HOPPER
BOX / BOXCAR / PLUG DOOR / HI CUBE → BOXCAR
GONDOLA / GON → GONDOLA
FLAT / FLATCAR → FLAT
REFRIGERATOR / REEFER → REFRIGERATOR

If RAPID does not send equipment type, or sends an unmapped type, the current default is TANK. That default should be reviewed carefully before expanding automation to new customer/product lanes.

The response includes how the type was matched:

rapid_equipment_type
default_missing_rapid_equipment_type
default_unmapped_rapid_equipment_type

Relationship to AEI Readers

RAPID and AEI answer different questions.

RAPID = business/order/waybill context
AEI   = physical reader detection
Trackage = yard inventory state
BIRCH = work-order and billing/work context

A strong operational picture comes from reconciling all of them:

RAPID says what car/order is expected and what the waybill context is.
AEI says a car physically passed a reader.
Trackage says whether the yard believes the car is active on property.
BIRCH says whether work and billing context exists.

Example high-confidence case:

RAPID sends a valid car/order
AEI sees the same car inbound
Trackage does not already show the car active
BIRCH has no active duplicate

Example exception case:

RAPID sends a car that already has an active BIRCH work order

That should represent/update the existing order, not create a duplicate.

Operational safeguards

Important safety behavior in the current system:

API keys are hashed before lookup.
Inactive API clients are rejected.
Duplicate unchanged waybill payloads are skipped.
Changed waybills preserve previous rows as history.
BIRCH order creation runs inside a transaction.
Active BIRCH railcars are represented instead of duplicated.
Work order numbers are allocated while the workorder table is locked for update.
Full raw RAPID payloads are retained for audit/debugging.

Troubleshooting checklist

When RAPID data does not appear or looks wrong:

Confirm the RAPID source is sending to the correct endpoint.
Confirm the request includes x-api-key.
Check whether the API client is active in the RAPID client table.
Confirm each waybill has waybill_id and source.
Check whether the payload was skipped because it matched the existing raw payload.
Inspect the current row and history rows for the waybill ID.
For BIRCH order creation, verify the railcar ID normalizes to 10 characters or fewer.
Check whether an active BIRCH work order already exists for the railcar.
Verify the model work order still exists and has the expected template content.
Review API server logs for [RAPID] messages.

Never paste RAPID API keys, Supabase service keys, database passwords, or raw credential-bearing URLs into chat, docs, commits, memory, or task logs.

Maintenance notes

Treat model work order changes as production-affecting.
Review equipment-type mappings before adding new traffic lanes.
Keep raw payload retention because it is the main audit/replay mechanism.
Prefer adding explicit mapping rules over relying on default TANK fallback.
When changing the ingest schema, update the viewer and documentation together.
When changing BIRCH model-order behavior, test inserted and represented paths.

What RAPID contributes​

Current production endpoints​

High-level flow​

Authentication model​

Waybill ingest​

BIRCH model-order endpoint​

Waybill ingest behavior​

Required fields​

Current-record model​

Dedupe behavior​

Stored fields​

RAPID viewer behavior​

BIRCH model-order import​

Railcar identity​

Existing active work order behavior​

New BIRCH order behavior​

Returned action values​

Equipment-type translation​

Relationship to AEI Readers​

Operational safeguards​

Troubleshooting checklist​

Maintenance notes​