Construct edge AI brokers with the AWS IoT Greengrass Element SDK for Rust

As Web of Issues (IoT) deployments scale to thousands and thousands of sensors producing steady knowledge streams, processing every thing within the cloud turns into impractical due to latency, bandwidth value, and connectivity constraints. Edge AI brokers resolve this by classifying sensor anomalies domestically on resource-constrained units and escalating solely complicated reasoning to the cloud. The AWS IoT Greengrass Element SDK for Rust makes this potential on units with tight reminiscence budgets (below 256 MB RAM), with a runtime footprint below 0.5 MB.

Paired with a purpose-built Open Neural Community Change (ONNX) classification mannequin, Rust-based AWS IoT Greengrass elements run AI inference domestically whereas leaving the vast majority of gadget reminiscence accessible for different processes. ONNX offers a compact, optimized mannequin format designed for environment friendly inference throughout {hardware} platforms, together with ARM and x86 CPUs, GPUs, and devoted edge accelerators.

On this submit, we describe the structure and design selections for constructing a Rust-based AWS IoT Greengrass part that runs a quantized ONNX anomaly classification mannequin for native sensor knowledge evaluation on industrial gateways. We clarify why we selected Rust over Python, ONNX over a general-purpose mannequin, and a bounded offline queue. We additionally embrace a part recipe excerpt and deployment instructions so you possibly can reproduce the sample. When the native mannequin identifies a posh anomaly requiring deeper evaluation, the part escalates to a cloud-based agent operating on Amazon Bedrock AgentCore. Escalation travels via AWS IoT Core Message Queuing Telemetry Transport (MQTT) messaging.

Use case

A water utility operates 2,000 distant pump stations throughout a rural distribution community. Every station has a microcontroller-class gateway (ARM Cortex-A53, 256 MB RAM, intermittent mobile connectivity) monitoring move fee, stress, vibration, and temperature sensors.

At these distant websites, an undetected pump fault can escalate into an outage that’s costly and gradual to restore. Crews typically journey hours to achieve a station, and a single truck roll can take days to schedule, so a failure caught late prices far a couple of caught early. Early, on-site detection is what retains a minor anomaly from turning into a multi-day service disruption.

To ship that early detection given the constraints of every web site, the answer should meet the next necessities:

• Classify sensor anomalies domestically inside 200 ms, as a result of a pump exhibiting early failure indicators should be flagged quick sufficient to behave earlier than injury compounds. A round-trip to the cloud over intermittent mobile hyperlinks is each too gradual and unavailable throughout outages.
• Function throughout community outages (mobile connectivity drops for hours throughout storms), when native detection is the one line of protection.
• Escalate complicated multi-sensor correlations to a cloud agent for root trigger evaluation and work order era.
• Maintain per-gateway reminiscence utilization below 64 MB for the AI part. The gateway’s 256 MB RAM is shared with the working system, the Greengrass nucleus, and the prevailing supervisory management and knowledge acquisition (SCADA) and telemetry processes that should hold operating.

Structure

The next diagram reveals the edge-to-cloud AI sample with the Rust Greengrass part.

Determine 1: Edge-to-cloud AI structure for the Rust Greengrass part

The structure spans two zones: the sting pump station gateway and the AWS Cloud. The sting gateway hosts the Rust Greengrass part and the ONNX classifier. The cloud hosts AWS IoT Core, IoT guidelines, AWS Lambda, Amazon Bedrock AgentCore runtime, and Amazon DynamoDB for historic telemetry storage. Amazon Easy Storage Service (Amazon S3) shops the binary and mannequin artifacts for fleet deployment.

The next walkthrough describes the information move via the structure:

1. Industrial sensors publish readings via Modbus (an industrial communication protocol) to the gateway. The Rust Greengrass part receives them via native inter-process communication (IPC) at 1-second intervals.
2. The Rust part buffers a full 60-second sensor window, runs it via the quantized ONNX classifier, and produces a confidence-scored anomaly classification. The repository ships a pattern mannequin for testing. For manufacturing workloads, scale the mannequin to your classification complexity.
3. The part publishes telemetry, alerts, and escalation messages to AWS IoT Core via an MQTT queue. Throughout community outages, messages queue domestically in first-in, first-out (FIFO) order (as much as 1,000 messages) and drain on reconnect.
4. AWS IoT Core routes messages to 2 IoT guidelines. The escalation rule forwards complicated anomalies to an AWS Lambda perform, which invokes a Strands Brokers-based agent on AgentCore runtime.
5. The agent queries Amazon DynamoDB for 7-day historic sensor baselines at that station to tell its root trigger evaluation.
6. The telemetry IoT rule performs a direct write of sensor knowledge to Amazon DynamoDB (90-day time to reside) for historic storage. No Lambda perform is required on this path.
7. The agent publishes a suggestion (severity, possible trigger, really helpful motion) again to the Rust part via AWS IoT Core MQTT.
8. The Rust part shows the advice as an area alert on the Human-Machine Interface (HMI).
9. AWS IoT Greengrass pulls binary and mannequin artifacts from Amazon S3 and deploys them to the sting gadget throughout fleet deployments.

This move locations classification on the edge (steps 1–2) as a result of the utility’s 200 ms latency goal is quicker than a round-trip to the cloud over intermittent mobile connectivity. A cloud-only design would add community latency to each classification and cease working solely through the outages these stations repeatedly expertise. The part escalates to the cloud (steps 4–5) just for the small fraction of anomalies that want historic correlation. This retains the reasoning-heavy work the place compute is plentiful whereas time-sensitive selections keep native.

Selecting between the Python SDK and Rust SDK

AWS IoT Greengrass helps part growth in a number of languages. The AWS IoT Greengrass Element SDK offers IPC APIs in C, C++, and Rust. For Python and Java elements, use the AWS IoT Machine SDK (v2) for Greengrass IPC. For AI workloads on constrained units (below 256 MB RAM), the minimal footprint of the Rust Element SDK leaves extra reminiscence accessible for the mannequin and inference runtime. For units with extra accessible reminiscence, the Python path (via the Machine SDK) offers sooner growth iteration and entry to the broader set of Python ML libraries.

The next desk summarizes the traits related to constrained-device AI workloads. Values are based mostly on testing with the pattern workload within the sample-greengrass-rust-edge-ai-agent. See the repository’s benchmarks/ listing for copy steps.

For the pump station use case (64 MB finances for the AI part), the Rust SDK is the suitable selection. For units with 1+ GB RAM the place growth velocity is prioritized, the Python SDK stays the sooner path to manufacturing.

Implementation

This part walks via the design of the answer, beginning with the 2 deployable artifacts. It then covers the sting part’s process construction, mannequin choice for constrained units, the offline-first sample, and the cloud agent design.

You deploy two artifacts:

1. Rust Greengrass part – Cross-compiled for aarch64-unknown-linux-gnu (ARM64), deployed utilizing a Greengrass deployment to the gadget fleet. Incorporates the inference binary, ONNX mannequin file, and part recipe.
2. AWS Cloud Growth Package (AWS CDK) – Deploys the AWS IoT Core guidelines, AWS Lambda perform, AgentCore harness (a functionality of Amazon Bedrock AgentCore), Amazon DynamoDB desk (telemetry, 90-day time to reside (TTL)), and AWS Id and Entry Administration (IAM) roles.

Edge part design

The Rust part makes use of three logical duties operating concurrently utilizing tokio (Rust’s asynchronous runtime):

• Ingestion and inference process – Subscribes to native IPC subjects utilizing the aws-greengrass-component-sdk crate and buffers sensor readings in a sliding window (60 seconds). Classifies each full window utilizing the ort crate (Rust bindings for ONNX Runtime) and returns confidence-scored anomaly sorts (regular, single-sensor fault, multi-sensor correlation, unknown). As a result of readings arrive at 1 hertz (Hz) and inference completes in below 50 ms, ingestion and inference run sequentially in the identical process.
• Communication process – Publishes categorised alerts domestically or escalates to the cloud via MQTT. Handles offline queuing with a bounded FIFO queue (1,000 messages, drop-oldest on overflow) for intervals with out connectivity. We sure the queue at 1,000 messages as a result of unbounded buffering dangers exhausting the gateway’s restricted RAM throughout a chronic outage. Dropping the oldest messages first retains the newest anomaly state accessible when connectivity returns.
• Response process – Subscribes to the cloud suggestion matter and appends acquired suggestions to an area log file for HMI show.

The AWS IoT Greengrass Rust SDK offers synchronous C bindings via a overseas perform interface (FFI). Our part bridges these to the tokio async runtime utilizing channels, so IPC subscription callbacks feed the async ingestion loop with out blocking.

The part recipe is the configuration file that defines a Greengrass part’s lifecycle and dependencies. It specifies aarch64 because the goal structure and declares the ONNX mannequin as an artifact dependency saved in Amazon Easy Storage Service (Amazon S3). AWS IoT Greengrass downloads the mannequin throughout deployment and locations it at a identified filesystem path.

Mannequin choice for edge

For structured sensor knowledge classification (time collection anomaly detection throughout 4 sensor channels), a purpose-built ONNX mannequin is extra acceptable than a general-purpose language mannequin. A general-purpose language mannequin is way bigger than the reminiscence finances permits. It additionally provides inference latency that exceeds the 200 ms goal and provides no accuracy profit on fixed-schema numeric sensor knowledge. ONNX offers a compact, quantized mannequin that runs within the ort crate with no separate runtime to put in. The repository ships a pattern mannequin for testing. The next record describes its enter format, output lessons, structure, dimension, and measured inference latency:

• Enter: 60-second sliding window of 4 sensor channels (240 knowledge factors, channel-major format).
• Output: 4 lessons with softmax confidence (regular, single_sensor_fault, multi_sensor_correlation, unknown).
• Structure: 1D convolutional neural community (CNN) with Squeeze-and-Excitation channel consideration.
• Measurement: roughly 25,000 parameters, roughly 23 KB (enough for demonstration).
• Inference latency: lower than 50 ms on ARM Cortex-A53 (see the benchmarks/ listing within the sample-greengrass-rust-edge-ai-agent for copy steps).

For manufacturing workloads with extra complicated classification necessities, scale the mannequin structure. A ten million-parameter mannequin at int8 quantization produces roughly 12 MB, which inserts nicely inside the 64 MB reminiscence finances.

Prepare this mannequin utilizing Amazon SageMaker with historic sensor knowledge, export to ONNX format, and quantize for edge deployment.

Offline-first sample

The part operates totally offline for native classifications. Throughout community outages:

• Native classifications proceed with out interruption (the mannequin runs domestically, no cloud dependency).
• Cloud escalation messages queue within the bounded FIFO queue, which holds as much as 1,000 messages. When the queue overflows, the oldest messages drop.
• When connectivity returns, the communication process drains the queue so as, and the cloud agent processes backlogged escalations with timestamps intact.

The gadget doesn’t block on community availability for native security selections.

Cloud agent design

Amazon Bedrock AgentCore is a platform to construct, join, and optimize brokers at scale together with your selection of framework or mannequin. This answer runs a Strands Brokers-based agent on the AgentCore runtime. The agent makes use of two instruments:

• query_history – Queries Amazon DynamoDB for 7-day sensor baselines on the specified station and computes statistics (imply, normal deviation, development).
• publish_response – Publishes the advice again to the gadget via AWS IoT Core MQTT.

The DynamoDB telemetry desk is partitioned on thing_name with ts (ISO 8601 timestamp) as the type key, so the agent can range-query the final 7 days for a single station effectively.

The agent receives escalation messages containing the next knowledge:

• Sensor readings (60-second window).
• Native mannequin’s preliminary classification and confidence rating.
• Machine metadata (pump station ID, set up date, final upkeep).

The agent acts as a root trigger evaluation (RCA) agent. It queries the station’s historic sensor patterns, causes in regards to the root trigger, and generates a structured response containing severity degree, possible trigger, really helpful motion, and supporting proof from the historic knowledge.

The next instance reveals a structured RCA response that the agent publishes again to the gadget:

{
    "thing_name": "pump-station-0417",
    "severity": "HIGH",
    "probable_cause": "Bearing put on on principal pump motor",
    "recommended_action": "Schedule inspection inside 48 hours; monitor vibration development",
    "supporting_evidence": "Vibration RMS rose 38% above 7-day baseline; correlated temperature improve of 6°C",
    "confidence": 0.86
}

Area availability: The mannequin ID us.anthropic.claude-haiku-4-5-20251001-v1:0 is a US cross-Area inference profile that routes to US East (N. Virginia), US East (Ohio), and US West (Oregon). Availability may broaden to further AWS Areas. Alter the inference profile in case your workload requires a special Area.

Accountable AI issues: The agent’s suggestions are advisory. They show on the HMI for a human operator to overview earlier than initiating bodily actions (dispatching a technician, shutting down a pump). No automated actuation happens with out operator affirmation. For manufacturing deployments, we advocate configuring Amazon Bedrock Guardrails to constrain the agent’s response classes and forestall suggestions exterior the outlined motion set.

Fleet deployment and mannequin updates

With AWS IoT Greengrass deployments, you possibly can push the Rust binary and ONNX mannequin to the gadget fleet. Mannequin updates use the identical deployment mechanism. Add a brand new ONNX artifact to Amazon S3 and replace the part model within the recipe. Then create a brand new deployment focusing on the gadget group. AWS IoT Greengrass handles the rolling replace, together with rollback if the brand new part fails well being checks.

Key design selections

We selected Rust over Python for this part due to the runtime footprint distinction (lower than 0.5 MB in comparison with roughly 30 MB, as proven within the previous desk). This distinction determines whether or not the AI part suits inside a 64 MB gadget finances. This answer makes use of the usual AWS IoT Greengrass nucleus quite than Nucleus Lite. The Rust SDK runtime footprint (below 0.5 MB) suits inside Nucleus Lite’s 5 MB workload restrict. Nevertheless, the statically linked ONNX Runtime brings the whole binary to roughly 22 MB, which exceeds that ceiling. For deployments with out native inference (pure MQTT relay to cloud), Nucleus Lite is a viable different.

The next selections formed the remainder of the design:

• Escalation cooldown: Repeats of the identical anomaly sort are suppressed for 300 seconds per gadget. With out the cooldown, a caught sensor producing one studying per second produces 86,400 day by day escalations. With 300-second suppression, this reduces to roughly 288 (see the benchmarks/ listing within the repository for the calculation). The cooldown ledger resets when the anomaly sort modifications or the suppression window expires.
• Safety: The Rust binary communicates with AWS IoT Greengrass nucleus via native IPC (Unix area sockets), not community sockets. Cloud communication makes use of the gadget’s X.509 certificates managed by AWS IoT Greengrass. The ONNX mannequin file is integrity-checked at two levels: AWS IoT Greengrass verifies the S3 artifact digest at deployment time, and the part re-verifies the SHA-256 hash at startup earlier than loading the mannequin into reminiscence.
• Cross-compilation: The sample-greengrass-rust-edge-ai-agent features a multi-stage Dockerfile that cross-compiles the Rust binary with statically linked ONNX Runtime for aarch64-unknown-linux-gnu. This produces a self-contained binary with minimal runtime dependencies on the goal gadget (glibc 2.38+, libstdc++, libgcc_s).

The glibc dependency comes from ONNX Runtime’s dynamic linking necessities. A musl-libc static construct isn’t possible as a result of ONNX Runtime doesn’t assist musl. Confirm your goal gadget’s glibc model earlier than deploying, as some embedded Linux distributions (Yocto, Buildroot) might ship older variations.

Efficiency targets (sub-200 ms inference latency, below 30 MB peak RSS) are based mostly on the part design and mannequin sizing for ARM Cortex-A53 class units with 256 MB RAM. See the benchmarks/ listing within the repository for copy steps by yourself {hardware}.

Stipulations

To implement this answer, you want the next:

• An AWS account with AWS Cloud Growth Package (AWS CDK) bootstrapped within the goal Area.
• AWS IoT Greengrass core gadget (ARM64 Linux) with AWS IoT Greengrass nucleus 2.14 or later. Set interpolateComponentConfiguration to true within the nucleus configuration. This setting is required for {iot:thingName} interpolation in part configuration values. Lifecycle variable interpolation works with out this flag.
• Rust toolchain (1.89 or later) with aarch64-unknown-linux-gnu cross-compilation goal.
• Docker (for cross-compilation construct surroundings).
• AWS IoT Core configured with the Greengrass core gadget registered.
• Amazon Bedrock mannequin entry for Anthropic Claude Haiku 4.5 (us.anthropic.claude-haiku-4-5-20251001-v1:0).

Clone the sample-greengrass-rust-edge-ai-agent repository to get the entire implementation:

git clone https://github.com/aws-samples/sample-greengrass-rust-edge-ai-agent.git
cd sample-greengrass-rust-edge-ai-agent

Deploying

The deployment consists of two elements: the sting part deployed to your gadget fleet and the cloud stack deployed to your AWS account.

To deploy the sting part

1. Cross-compile the Rust part for ARM64 utilizing the supplied Dockerfile:

   docker construct 
     -f edge-component/Dockerfile 
     -t edge-ai-classifier-build 
     --target output 
     --output "sort=native,dest=edge-component/dist" 
     edge-component/

2. The part recipe defines the lifecycle and artifact dependencies. The next excerpt reveals the important thing sections:

   Manifests:
       - Platform:
           os: linux
           structure: aarch64
         Artifacts:
           - URI: s3://BUCKET/artifacts/edge-ai-classifier
           - URI: s3://BUCKET/artifacts/sample_model.onnx
         Lifecycle:
           run:
             script: >-
               {artifacts:path}/edge-ai-classifier
               --model-path {artifacts:path}/sample_model.onnx
               --thing-name {iot:thingName}

3. Add the binary and mannequin artifacts to your Amazon S3 bucket in order that AWS IoT Greengrass can retrieve them throughout deployment:

   aws s3 cp edge-component/dist/edge-ai-classifier s3://BUCKET/artifacts/edge-ai-classifier
   aws s3 cp mannequin/sample_model.onnx s3://BUCKET/artifacts/sample_model.onnx

4. Create an AWS IoT Greengrass deployment focusing on your gadget group:

   aws greengrassv2 create-deployment 
     --target-arn arn:aws:iot:us-east-1:ACCOUNT:thinggroup/PumpStations 
     --components '{"com.instance.EdgeAIClassifier": {"componentVersion": "1.0.1"}}'

5. Run the fleet simulator from the repository to seed artificial sensor knowledge for testing:

   python scripts/simulate_sensors.py --devices 10 --duration 300

To deploy the cloud stack

1. Run cdk deploy from the cloud stack listing.
2. Word the CDK stack outputs (MQTT subjects and IoT rule configuration).

The sample-greengrass-rust-edge-ai-agent incorporates the entire implementation: Rust part supply, ONNX mannequin integration utilizing the ort crate, part recipe, cross-compilation Dockerfile, cloud CDK stack, and pattern classification mannequin.

Clear up

To keep away from ongoing prices, cease any operating simulators first. Every escalation invokes Amazon Bedrock and accrues value. Every escalation prices roughly $0.004 (assuming 1,500 enter tokens at $1/M and 500 output tokens at $5/M, utilizing Claude Haiku 4.5 pricing). In a practical check state of affairs the place 10 units common 8-12 escalations per day (not fixed anomalies), anticipate roughly $10-15/month. A worst-case caught sensor hitting the 288 day by day escalation cap on all 10 units would value roughly $345/month. See the Amazon Bedrock pricing web page for present per-token charges.

Then delete the AWS IoT Greengrass deployment and take away the cloud stack with cdk destroy. The repository’s scripts/cleanup.sh automates this course of. See the cleanup script documentation for particulars. The Amazon DynamoDB telemetry desk makes use of a TTL coverage. Knowledge older than 90 days is deleted mechanically.

Conclusion

On this submit, you explored the structure and key design selections for a Rust-based AWS IoT Greengrass part. The part classifies sensor anomalies domestically utilizing ONNX inference and escalates complicated circumstances to Amazon Bedrock AgentCore within the cloud. The structured RCA response proven earlier illustrates the end-to-end escalation, from an area classification to a cloud-generated suggestion returned to the gadget.

With this sample, the pump station targets anomaly classification in below 200 ms with out cloud dependency. Throughout outages, the sample queues escalations domestically and drains them on reconnect. Multi-sensor correlations path to the cloud agent for root trigger evaluation in opposition to 7-day historic baselines.

This submit makes use of a water utility pump station. The identical edge-to-cloud inferencing move applies to different workloads that want quick native classification with selective escalation to cloud reasoning. A predictive upkeep system on a producing line classifies gear vibration signatures domestically and escalates ambiguous failures for root trigger evaluation. In good buildings, the part detects HVAC or vitality anomalies on the edge and escalates cross-system correlations. Fleet and logistics telematics flag automobile sensor faults on the gadget and escalate multi-signal patterns for diagnostics. In every case, the Rust part retains latency-sensitive selections native whereas routing the reasoning-heavy work to the cloud solely when wanted.

Clone the sample-greengrass-rust-edge-ai-agent and deploy it to your individual edge units utilizing the recipe and instructions proven on this submit.

References

Concerning the authors