How to Filter SQS Messages So Each Consumer Gets Only Its Own

ReceiveMessage has no filter parameter. MessageAttributeNames chooses which attributes come back with each message. It does not choose which messages come back.

On a FIFO queue, MessageGroupId does not help either. It sets the order of messages. It does not let a consumer pick a group.

So filter before the messages reach the queue. Give each consumer its own queue, and put only the right messages in it.

Why filtering on receive fails

The obvious workaround is to have every consumer read the shared queue, keep the messages it wants, and leave the rest. Every receive hides the message and raises its ApproximateReceiveCount. SQS counts a receive without a delete as a failed receive. Enough receives by the wrong consumers push the message past maxReceiveCount and into the dead-letter queue. For the details, see SQS Essentials and How to View SQS Messages Without Deleting Them.

Lambda event filtering and EventBridge Pipes filters do not solve this. When an SQS message does not match the filter, Lambda and Pipes delete it from the queue. That saves work for one consumer, but the other consumers never see the message. Do not share one queue between several filtered consumers.

Send each message to the right queue

If the producer already knows which consumer a message is for, send it straight to that consumer's queue. You need SNS or EventBridge only when one message can go to several consumers, or when the producer should not know who reads it.

Option 1: SNS filter policies, one queue per consumer

Use this when a producer sends messages that different consumers care about.

The producer publishes to an SNS topic instead of the queue. Each consumer gets its own SQS queue, subscribed to the topic. Each subscription has a filter policy. SNS sends a message to a subscription only if the message matches its filter policy. A subscription with no filter policy gets every message.

1. Create one queue per consumer.
2. On each queue, add an access policy that lets the topic call sqs:SendMessage. Set Principal to sns.amazonaws.com and add an aws:SourceArn condition with the topic ARN.
3. Subscribe each queue with its own filter policy.

This subscription delivers only messages whose JSON body has "type": "order.shipped":

aws sns subscribe \
  --topic-arn arn:aws:sns:us-east-1:123456789012:orders \
  --protocol sqs \
  --notification-endpoint arn:aws:sqs:us-east-1:123456789012:shipping \
  --attributes '{
    "FilterPolicyScope": "MessageBody",
    "FilterPolicy": "{\"type\": [\"order.shipped\"]}",
    "RawMessageDelivery": "true"
  }'

Check these before you subscribe:

• Scope. FilterPolicyScope defaults to MessageAttributes. Then the policy matches the message attributes the producer set on publish, not the body. Set MessageBody to match body fields. SNS expects the body to be a well-formed JSON object.
• Raw delivery. Without RawMessageDelivery, SNS wraps each message in a JSON envelope, and your payload sits in its Message field. With it, the queue gets the body as published. Raw delivery sends at most 10 message attributes. SNS discards a message that has more.
• Encryption. An encrypted queue needs a customer managed KMS key, or no encryption at all. The key policy must let sns.amazonaws.com call kms:Decrypt and kms:GenerateDataKey.
• FIFO. A FIFO queue can subscribe only to a FIFO topic. Its name ends in .fifo.
• Delay. A new or changed filter policy can take up to 15 minutes to fully take effect. Test with real publishes before you switch consumers over.

Option 2: EventBridge rules, one queue per consumer

Use this when your producers already send events to EventBridge, or when you want to match on more than one field.

The producer calls PutEvents with a Source, a DetailType, and a JSON Detail. Each consumer gets a rule with an event pattern and its own queue as the target. EventBridge sends an event to the target only if the pattern matches.

aws events put-rule \
  --name shipping-orders \
  --event-pattern '{"source": ["com.example.orders"], "detail": {"type": ["order.shipped"]}}'

aws events put-targets \
  --rule shipping-orders \
  --targets "Id"="shipping-queue","Arn"="arn:aws:sqs:us-east-1:123456789012:shipping"

Check these before you add the target:

• Permission. The queue needs an access policy that lets events.amazonaws.com call sqs:SendMessage, with an aws:SourceArn condition set to the rule ARN. Or give the target an IAM role that has sqs:SendMessage.
• Encryption. An encrypted queue needs a customer managed KMS key, or no encryption at all. The console default (SSE-SQS) and the alias/aws/sqs key do not work. The key policy must let events.amazonaws.com call kms:Decrypt and kms:GenerateDataKey.
• Body. The queue gets the whole event as JSON, and your payload is under detail. Use an input transformer to send a different shape.
• FIFO. A FIFO target needs a message group ID in the target's SqsParameters, and the queue needs content-based deduplication turned on.

Option 3: Read the result by ID instead

Sometimes the question is really "how do I get the result for my job?" A client starts a job, a worker does it, and the client wants that one result. A queue is the wrong place to look for it.

Have the worker write the result to a store, keyed by job ID. For example, it puts an item in a DynamoDB table with jobId as the partition key. The client then calls GetItem with its job ID and gets that one item. It never receives anyone else's message.

GetItem is eventually consistent by default. If the client reads right after the worker writes, set ConsistentRead to true.

Keep using the queue to send work to the workers. Stop using it to send results back.

Check the result with Tarmac

After you split one queue into several, confirm that each message went where you meant it to.

Tarmac is a desktop client for SQS on macOS and Windows. Select a consumer queue and choose Queue Tools ▸ View Topology. The diagram lists the SNS topics that the queue's access policy allows. It does not show filter policies or EventBridge rules. To check the routing, fetch a few messages and read them. If a queue holds messages its consumer should not get, fix the filter policy or event pattern. See Receive and Cache AWS SQS Messages.