Introducing AWS IoT Core’s Direct Messaging: Decrease-cost server-to-device communication with higher observability

AWS IoT Core now helps direct messaging for point-to-point communication. Beforehand, sending a message to a single IoT system required publishing to an MQTT subject the system subscribed to, with no built-in supply affirmation from the system. With the brand new direct messaging functionality, you may ship a message to any system related to AWS IoT Core, decreasing messaging value in comparison with publish-subscribe (Pub-Sub) messaging for one-to-one communication patterns, and offering a supply acknowledgment from the IoT system. AWS IoT Core additionally makes use of the supply acknowledgment to supply detailed API response codes and emit Amazon CloudWatch Logs, so you will have visibility into message supply standing and failure causes.

Level-to-point communication between backend servers and IoT units is a standard sample in related system architectures. It exhibits up in use instances like sending firmware updates to a wise residence equipment, pushing transaction updates to a cost system, or controlling a wise automobile. For these one-to-one interactions, routing by means of a Pub-Sub message dealer isn’t environment friendly, since you’re not utilizing the fan-out profit that Pub-Sub was designed to supply. As well as, you obtain the message supply acknowledgment from the Pub-Sub dealer reasonably than from the IoT system, requiring you to construct your personal customized resolution for supply affirmation.

What’s Direct Messaging?

AWS IoT Core introduces the SendDirectMessage HTTP API. With this API, you may ship a message to a selected IoT system or consumer recognized by its MQTT consumer ID. Quite than publishing to an MQTT subject and counting on subscription matching, AWS IoT Core routes the message point-to-point to the goal system. You skip the subscription-matching layer, there’s no fan-out, and you’ve got the choice to obtain affirmation from the consumer.

The message is delivered to the system on its present MQTT connection (over TCP) to AWS IoT Core, with no device-side adjustments required. The API helps two modes:

1. With out supply affirmation – The message is delivered at MQTT High quality of Service (QoS) 0. The HTTP API returns 200 OK after AWS IoT Core accepts and dispatches the message.
2. With supply affirmation – The message is delivered at MQTT QoS 1. The HTTP API returns 200 OK solely after the system acknowledges receipt with an MQTT PUBACK (publish acknowledgment), offering true end-to-end supply affirmation. If the system doesn’t acknowledge inside the timeout, the API returns 504 Gateway Timeout.

Key advantages

Constructed-in supply affirmation

With supply affirmation enabled (affirmation=true), you obtain acknowledgment instantly from the system, not solely from the dealer. AWS IoT Core delivers the message at QoS 1 and waits for the system’s PUBACK earlier than returning 200 OK. This removes the necessity to construct customized acknowledgment logic.

Rapid offline system suggestions

With conventional Pub-Sub, a server publishing to a subject receives a profitable response no matter whether or not the system acquired the message. With Direct Messaging, if the goal system isn’t related, the API returns 404 Not Discovered instantly. The HTTP response message and Amazon CloudWatch Logs describe the particular motive. For instance, when the response message states that the goal consumer ID isn’t related however has an lively persistent session, the system has an unexpired persistent session however is at present offline.

Value optimization

When you use direct messaging with out supply affirmation, you pay for a single Direct Message as a substitute of separate publish-in and publish-out operations. When you use direct messaging with supply affirmation, you pay for a single Direct Message with Affirmation as a substitute of separate publish-in, publish-out, and publish-ack operations. For particulars, see the AWS IoT Core pricing web page.

No device-side adjustments

Direct messages are delivered on present MQTT connections already established with AWS IoT Core. No firmware updates, SDK adjustments, or new subject subscriptions are required. In case your system already subscribes to the goal subject, Direct Messaging works instantly. In case your system doesn’t subscribe to the subject, confirm that your MQTT consumer library doesn’t filter messages on unsubscribed subjects. Most manufacturing shoppers (together with the AWS IoT Machine SDKs) deal with this accurately.

Use instances

Direct Messaging is the best alternative when:

• It is advisable to ship a message to a selected system (not a multicast).
• It is advisable to know whether or not the system truly acquired the message.
• You wish to simplify retry and error-handling logic.

Examples: server-to-device instructions (locking a automobile, toggling a wise equipment, pushing a configuration replace), system acknowledgments, cost-sensitive high-volume messaging, and real-time notifications.

Conventional Pub-Sub stays the best alternative while you want fan-out (one message to many subscribers) or message queuing for offline units.

Getting began

On this submit, you’ll configure the AWS Id and Entry Administration (IAM) insurance policies, ship a Direct Message in three other ways, and obtain it on a related system.

Stipulations

You could have the next conditions to observe together with this submit.

• An AWS account with AWS IoT Core configured.
• IoT units registered and related by means of MQTT 3.1.1 or MQTT 5.0. See the AWS IoT Core Developer Information to discover ways to join units.
• Backend server code that calls the Direct Messaging API (server-side adjustments solely).

Step 1: Configure authorization

Each the sender and the receiver require particular coverage actions. The sender wants iot:SendDirectMessage permission with the goal consumer’s Amazon Useful resource Identify (ARN) because the useful resource. You’ll be able to optionally limit which subjects can be utilized with the iot:Subject situation key.

The next sender coverage permits direct messages to consumer myDevice on the subjects instructions/reboot and instructions/replace. For SigV4-authenticated backend servers, add this to an IAM coverage. For X.509-authenticated units, add it to an AWS IoT Core coverage.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": "iot:SendDirectMessage",
            "Resource": "arn:aws:iot:us-west-2:123456789012:client/myDevice",
            "Condition": {
                "StringEquals": {
                    "iot:Topic": ["commands/reboot", "commands/update"]
                }
            }
        }
    ]
}

The receiver will need to have iot:Obtain permission on the goal subject. The receiver doesn’t want iot:Subscribe. Direct Messaging delivers to the related consumer with out requiring a subject subscription.

The next receiver coverage permits iot:Obtain on two particular subjects:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": "iot:Receive",
            "Resource": "arn:aws:iot:us-west-2:123456789012:topic/commands/reboot"
        },
        {
            "Effect": "Allow",
            "Action": "iot:Receive",
            "Resource": "arn:aws:iot:us-west-2:123456789012:topic/commands/update"
        }
    ]
}

You may also use a wildcard to obtain direct messages on any subject beneath a prefix:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": "iot:Receive",
            "Resource": "arn:aws:iot:us-west-2:123456789012:topic/commands/*"
        }
    ]
}

For extra coverage examples, together with sending to any consumer on particular subjects or on any subject, see Direct messaging coverage examples.

Step 2: Ship a Direct Message

Senders make an HTTP POST request to a client-specific URL:

POST https://{IoT_data_endpoint}/connections/{url_encoded_client_id}/messages?subject={url_encoded_topic}&affirmation=true&timeout=10

{IoT_data_endpoint} is your account’s AWS IoT system knowledge endpoint. The next three examples ship the identical instructions/reboot message to consumer myDevice with supply affirmation.

Utilizing curl (X.509 consumer certificates authentication, port 8443):

curl --tlsv1.2 
  --cacert AmazonRootCA1.pem 
  --cert system.pem.crt 
  --key non-public.pem.key 
  --request POST 
  --data '{"motion": "reboot"}' 
  "https://{IoT_data_endpoint}:8443/connections/myDevice/messages?subject=commandspercent2Freboot&affirmation=true&timeout=10"

Utilizing the AWS Command Line Interface (AWS CLI):

aws iot-data send-direct-message 
  --client-id myDevice 
  --topic instructions/reboot 
  --confirmation 
  --timeout 10 
  --payload '{"motion": "reboot"}' 
  --cli-binary-format raw-in-base64-out 
  --region us-west-2 
  --endpoint-url https://{IoT_data_endpoint}

The --cli-binary-format raw-in-base64-out choice is required with AWS CLI v2 so the --payload worth is shipped as-is. To make it the default, run aws configure set cli-binary-format raw-in-base64-out. This command requires AWS CLI v2 model 2.34.57 or newer.

Utilizing the AWS SDK for Python (Boto3):

import boto3
import json

consumer = boto3.consumer("iot-data", region_name="us-west-2")

response = consumer.send_direct_message(
    clientId="myDevice",
    subject="instructions/reboot",
    affirmation=True,
    timeout=10,
    contentType="utility/json",
    payload=json.dumps({"motion": "reboot"}).encode("utf-8"),
)

# The response accommodates a standing message and a traceId for troubleshooting.
print(response["message"], response["traceId"])

With affirmation=true, the API waits for the system to acknowledge. A 200 OK confirms end-to-end supply. A 504 Gateway Timeout signifies the receiver didn’t acknowledge inside the timeout interval. The supply state is ambiguous, so implement idempotent dealing with for those who retry.

Request parameters:

Step 3: Obtain Direct Messages on the system

Your present MQTT consumer receives Direct Messages on the required subject. In contrast to customary MQTT Pub-Sub, the system doesn’t want an lively subscription. Direct Messages are delivered primarily based on the consumer ID. The system can nonetheless optionally subscribe to the subject, but it surely’s not required for supply.

The QoS degree of the delivered message is about by the sender’s affirmation parameter, not by the receiver’s subscription. When affirmation=true, the message arrives at QoS 1 and the consumer should ship a PUBACK to acknowledge supply. Most MQTT consumer libraries do that robotically. When affirmation=false, the message arrives at QoS 0 with no acknowledgment required. Make sure that your consumer handles each QoS 0 and QoS 1 incoming messages accurately.

Monitoring with CloudWatch

AWS IoT Core returns detailed HTTP response codes for each SendDirectMessage name. If you allow AWS IoT Core CloudWatch logging, the service additionally emits SendDirectMessage occasion logs that embody a machine-readable motive area, so you may construct automated retry logic, monitor system connectivity, and troubleshoot supply points programmatically. Evaluate the HTTP response message or CloudWatch logs to establish the particular motive for any failure.

HTTP response standing codes:

Limitations and issues

• No message queuing – Direct Messages aren’t queued for offline units. Use Pub-Sub with QoS 1 and chronic periods for message persistence.
• No message retention – The MQTT retained flag isn’t supported. Use Amazon DynamoDB or AWS IoT Machine Shadow for state synchronization.
• No Guidelines Engine processing – The AWS IoT guidelines engine doesn’t course of Direct Messages.
• Reserved subjects – Direct Messaging helps customized subjects. Subjects should not begin with $ and should not be AWS IoT reserved subjects.
• Consumer ID restrictions – Consumer IDs should not exceed 128 characters and should not begin with $. Consumer IDs containing characters which might be invalid in HTTP requests (equivalent to /) should be percent-encoded within the URL path.
• Limits and quotas – See the AWS IoT Core developer information and repair quotas for payload dimension, subject depth, and API limits.
• Protocol – Works with present MQTT 3.1.1 and MQTT 5.0 shoppers. (MQTT 5.0 properties equivalent to content material kind and response subject should not delivered to MQTT 3.1.1 shoppers.)

Availability

AWS IoT Core Direct Messaging is on the market at this time in all AWS Areas the place AWS IoT Core is on the market.

Conclusion

On this submit, you noticed how AWS IoT Core Direct Messaging routes server-to-device messages point-to-point by MQTT consumer ID, returns a real end-to-end supply acknowledgment while you decide in with affirmation=true, and stories actionable HTTP response codes for offline or unauthorized targets. The aptitude works on present MQTT connections, so you may undertake it with out modifying system firmware.

To get began, see the Direct Messaging subject within the AWS IoT Core Developer Information. For pricing particulars, see the AWS IoT Core pricing web page. Check in to the AWS IoT console to start out sending direct messages to your related units.

Concerning the authors