Comparing Azure Queues and Service Bus Queues services

Microsoft Azure supports two types of queuing mechanisms: Azure Queues and Service Bus Queues .

Azure Queues , is part of the Azure storage infrastructure, providing a simple REST-oriented Get / Put / Peek interface for reliable, guaranteed delivery of messages within a service or between several services.

Service Bus Queues is part of the Azure messaging infrastructure, which deals with queuing, publishing / subscribing, web services and integration patterns (more about this Overview of Service Bus Messaging Patterns ).
')
Services run in parallel, Azure Queues appeared earlier as a separate queue mechanism built on top of Azure storage services.
Service Bus Queues are built on top of the “message broker” infrastructure, designed to connect applications or application components that may require support for various protocols, work in different domains or network environments.

This article compares these two queuing systems, focusing on differences in their behavior and implementation. The article also gives advice on choosing the type of queue, which is better suited to your application.

Considerations on the choice of technology

Both Azure Queues and Service Bus Queues are queue services on the Microsoft Azure platform. They have a slightly different set of features, so you can use one of them, or the other, or both together - depending on what your application needs. Below are recommendations for selection.

So, you should choose Azure Queues , if:

• Your application needs to store more than 80 GB of data in one queue, and each message exists for no more than 7 days.
• Your application needs to monitor the processing status of the message in the queue. For example, you want to be aware of the fall of the worker who processed the message, and when this event occurs, transfer further processing to another worker.
• You need logs of all transactions in your queue.

You should choose Service Bus Queues if:

• You need to be able to receive messages without a permanent polling. Service Bus Queues supports TCP-protocol, allowing you to quickly receive information about new messages.
• You need to ensure that the message handling order (FIFO) is maintained.
• You have experience with Service Bus for Windows Server and you want to use it when working with queues in Azure.
• You want to automatically detect duplicate messages.
• You want in your application to process messages in parallel in the so-called “threads” (messages are associated with the stream by SessionId ). In this model, each application node receiving messages is responsible for processing one stream. When a thread is transferred to the processing node, the latter can work with the stream using transactions.
• Your application needs to be transactional when sending or receiving a group of messages.
• The lifetime (TTL) of a message can exceed 7 days.
• Your application needs to process messages larger than 64 KB (but no more than 256 KB).
• You need to provide a role-based security model, to implement different levels of access rights for both providers and consumers of messages.
• The total size of all messages in the queue does not exceed 80 GB.
• You want to use the AMQP 1.0 protocol for working with messages. More information here - Service Bus AMQP Overview .
• You assume the theoretical possibility of transition from the “one supplier, one recipient” model to a model, where messages can be generated by several suppliers and redirected to more than one recipient (possibly, according to some filtering rules).
• You need a “at-most-once” type of delivery guarantee and you don’t want to build a special infrastructure for this.
• You need the ability to batch send and receive messages.
• You need full integration with the Windows Communication Foundation (WCF) stack and the .NET Framework.

Comparing Azure Queues and Service Bus Queues

The tables in the following sections are divided into logical groups, each of which compares Azure Queues and Service Bus Queues in some parameters.

Fundamental features

Comparison criterion	Azure queues	Service Bus Queues
Guaranteed to maintain order	Not	Yes - FIFO (through the use of sessions)
Delivery guarantee	"At least once"	"At least once" “Maximum once”
Transaction support	Not	Yes (through the use of local transactions)
Method of obtaining	Not blockable (ends immediately if there are no new messages)	Lockable with or without timeout (see “ Comet technique ”) Not blockable (through the use of the .NET API)
Pushing messages by the server to the client	Not	Yes OnMessage and OnMessage sessions .NET API
Method of obtaining	Peek & Lease	Peek & lock Receive & Delete
Exclusive access to the message	Based on Lease	Based on lock
Lease / Lock timeout duration	30 seconds (default) 7 days (maximum) (can be updated or reset using the UpdateMessage API)	60 seconds (default) (can be updated or reset using the RenewLock API)
Lease / Lock Granularity	At the message level (each message can have its own timeout, for each message it can be changed using the UpdateMessage API)	At the queue level (each queue has its own timeout, which applies to all messages in this queue, the timeout can be updated using the RenewLock API)
Batch receipt	Yes (when receiving it is explicitly indicated how many messages are required, maximum - 32)	Yes (implicitly through the inclusion of preloading or explicitly through transactions)
Batch shipping	Not	Yes (via transaction or batch sending on client side)

Additional Information

• If you already use Azure Storage Blobs and Tables and start using queues, you still have guaranteed availability of services within 99.9%. If you use Blobs and Tables with Service Bus Queues, reliability will be less.
• Guaranteed Message Delivery (FIFO) in the Service Bus Queues requires the use of transactions. If an application crashes while the message is in Peek & Lock mode, after the application is restarted, it will receive the same message as soon as the TTL expires.
• Azure Queues is designed to support standard queuing scenarios, such as splitting application components to increase scalability and resilience to individual component failures.
• Service Bus Queues supports the At-Least-Once delivery guarantee model. In addition, the “Maximum Once” model can be implemented with the help of transactions and atomic receipt of messages with updating the session state. Azure Workflow service uses this technique to provide its “Maximum Once” warranty.
• Azure Queues provides a unified and consistent access interface, similar to the access interface to tables and blobs, which can be convenient if you need to use all these services.
• Service Bus Queues provides local transaction support in the context of each queue.
• The “Receive and Delete” mode supported by Service Bus Queues allows reducing the number of operations for receiving messages in exchange for a reduced guarantee of their receipt.
• Azure Queues allows you to handle the amount of message processing timeout. A worker can quickly process “short” messages, and if he suddenly falls, messages will be promptly transferred to another worker. At the same time, if the worker just needs more time to process the message, it can increase its timeout, making it clear that the message is still being processed.
• Azure Queues allow you to set a timeout "invisibility" of the message when it is published. The message will be available to the consumer only after this timeout expires. Service Bus Queues also allows you to set such a timeout, but only at the level of the queue metadata.
• The maximum timeout for blocking the operation of receiving messages in the Service Bus Queues is 24 days. However, when using the REST interface, this timeout is 55 seconds.
• Client-side batch operations provided by Service Bus Queues allow multiple messages to be combined into a single send operation. Batch send is available only for an asynchronous send operation.
• Supporting up to 200 TB of data in Azure Queues queues (and even more in the case of account virtualization) and an unlimited number of queues makes this service an ideal platform for SaaS providers.
• Azure Queues provides a flexible and efficient access delegation mechanism.

Additional features

Comparison criterion	Azure queues	Service Bus Queues
Scheduled Delivery	Yes	Yes
Automatic labeling of "dead" messages	Not	Yes
Increase the lifetime of the message in the queue	Yes (update timeout separately for each message in the queue)	Yes (separate API function)
Support for not fully processed messages	Yes	Yes
Post update	Yes	Yes
Server side transaction log	Yes	Not
Storage metrics	Yes Minute metrics: real-time for availability, TPS, number of API calls, number of errors, etc., see About Storage Analytics Metrics	Yes ( GetQueues method)
State management	Not	Yes Microsoft.ServiceBus.Messaging.EntityStatus.Active , Microsoft.ServiceBus.Messaging.EntityStatus.Disabled , Microsoft.ServiceBus.Messaging.EntityStatus.SendDisabled , Microsoft.ServiceBus.Messaging.EntityStatus.ReceiveDisabled
Auto message forwarding	Not	Yes
Clearing the entire queue	Yes	Not
Message groups	Not	Yes (via session posts)
Application status for a group of messages	Not	Yes
Duplicate detection	Not	Yes (configurable on the sender side)
WCF integration	Not	Yes (WCF - binding provided)
Integration with WF	External (requires building your own WF activities)	Built (provided by WF activities)
Message Viewer Groups	Not	Yes
Selection of messages of certain messages by session ID	Not	Yes

Additional Information

• Both technologies of queues allow to postpone delivery of the message for a certain time.
• Automatic message redirection allows you to collect messages from thousands of queues into one queue. You can use this mechanism to ensure security, control processes and isolate message providers from each other.
• Azure Queues supports the ability to change message content. You can use this to update the progress of the task corresponding to this message, so that, if necessary, show this information to the user or continue processing from the desired stage, rather than first. In Service Bus Queues, you can implement a similar scenario using sessions. Session allows you to save and retrieve application state (using the SetState and GetState methods ).
• The dead letter handling mechanism, which is supported by Service Bus Queues, allows you to isolate messages that cannot be successfully processed by the recipient or that have expired (TTL). Such messages will be moved to a special queue called $ DeadLetterQueue.
• A partially similar mechanism can be implemented in Azure Queues on the application side — by checking the DequeueCount property, you can redirect certain messages to another queue.
• Azure Queues allows you to get a detailed log of all transactions for each queue, as well as some aggregated metrics. Both features are very useful for debugging and understanding how your applications use the queue. They also help optimize performance and minimize costs when using queues.
• The concept of "message sessions", supported by Service Bus Queues, allows each message to belong to a logical group that can be associated with an individual recipient. You can use this functionality by filling in the SessionID field for each message. The recipient can then only request messages with a specific SessionID from the queue.
• The duplicate detection mechanism supported by the Service Bus Queues automatically removes messages with the same MessageID from the queue.

Capacity and quotas

Comparison criterion	Azure queues	Service Bus Queues
Maximum queue size	200 TB (limited by total storage for one account)	From 1 GB to 80 GB (determined at the stage of creating a queue)
Maximum message size	64 KB (48 KB using Base64 encoding) Azure supports large messages by combining queues and blobs - so you can store up to 200 GB of data in a single message	256 KB (including header and message body, maximum header size is 64 KB)
Maximum message lifetime	7 days	Not limited
Maximum number of queues	Not limited	10,000 (per namespace, can be increased)
Maximum number of customers	Not limited	Not limited (100 connections for TCP clients)

Additional Information

• For Azure Queues, if the message content contains characters that cannot be stored in XML, it must be encoded in Base64 . Please note that in this case the size of the payload in the message should not exceed 48 KB.
• For Service Bus Queues, each message consists of a header and body. The total size of the message should not exceed 256 KB.
• When customers use Service Bus Qeues via TCP, the total number of active connections to one queue should not exceed 100. Both senders and recipients are counted. If this quota is reached, subsequent requests will be rejected. This limit does not apply to customers who use the queue through the REST API.
• Service Bus Queues has limits on queue size. They are set when creating a queue and can be from 1 to 80 GB. If the limit is reached, the next message will not be accepted into the queue. More information - Azure Service Bus Quotas .
• If you need more than 10,000 Service Bus Queues in the same namespace, you can contact Azure technical support and ask for a limit increase for you. But it may be easier and faster to create additional namespace through the Microsoft Azure Management Portal.

Management and access

Comparison criterion	Azure queues	Service Bus Queues
Control protocol	REST over HTTP / HTTPS	REST over HTTPS
Transaction protocol	REST over HTTP / HTTPS	REST over HTTPS AMQP 1.0 Standard (TCP with TLS)
.NET API	Yes (.NET Storage Client API)	Yes (.NET API for message brokers)
Native c ++	Yes	Not
Java API	Yes	Yes
PHP API	Yes	Yes
Node.js API	Yes	Yes
Support for custom metadata	Yes	Not
Queue Naming Rules	Up to 63 characters (letters must be lowercase)	Up to 260 characters (queue names are case-insensitive)
Getting the number of messages in the queue	Yes (approximate, excluding messages for which TTL has expired and they are in the process of being deleted)	Yes (exact)
The function of viewing messages in the queue without their selection	Yes	Yes

Additional Information

• Azure queues provides the ability to add an arbitrary amount of metadata in the form of key / value pairs to the queue.
• Both queuing technologies allow you to view messages in the queue without modifying / deleting them, which can be useful for implementing queuing tools.
• Service Bus supports TCP-based .NET API, which works more efficiently than REST requests or AMQP 1.0 protocol.
• The queue name in Azure Queue can be from 3 to 63 characters long, can include letters in different case, numbers and hyphens. More information - Naming Queues and Metadata .
• The queue name in the Service Bus Queue can be up to 260 characters and can include the characters '.', '-' and '_' in addition to letters and numbers.

Performance

Comparison criterion	Azure queues	Service Bus Queues
Maximum throughput	Up to 2000 messages per second (for 1 KB messages)	Up to 2000 messages per second (for 1 KB messages)
Average delay	10 ms (with TCP Nagle disabled)	20-25 ms
Overload behavior	HTTP code 503 (request is not paid)	HTTP code 503 (request is not paid)

Additional Information

• One Azure Queue can handle up to 2000 transactions per second. A transaction is a Put , Get, and Delete operation. Sending one message to a queue ( Put ) is considered one transaction, but receiving a message is often a two-step operation — you must first receive a message ( Get ) and then delete it from the queue ( Delete ), which results in two transactions. Do not forget that in order to reduce the number of transactions you can receive in one request up to 32 messages, although you will have to remove them from the queue anyway. You can create more queues to increase bandwidth (the total number is unlimited).
• When your application tries to process more than 2000 messages per second, you will receive an “HTTP 503 Server Busy” response. In this case, you should reduce the frequency of calls to the queue.
• The delay in Azure Queues is on average 10 ms when processing messages up to 10 KB when working from an Azure virtual machine located in the same region as the queue
• Both queue services reject requests to queues in case of exceeding the maximum possible bandwidth. Both services do not consider such requests paid.
• Service Bus Queues performance tests have shown that one queue can handle up to 2000 messages per second at a message size of 1 KB. To provide more bandwidth, use multiple queues. More information - Best Practices for Performance Improvement Service Bus Brokered Messaging .
• When maximum throughput is reached when using Service Bus Queues, you will get a ServerBusExException (when using the .NET API) or “HTTP 503 Server Busy” (when using the REST API).

Authentication and authorization

Comparison criterion	Azure queues	Service Bus Queues
Authentication	By symmetric key	By symmetric key
Access control model	Delegated access to SAS tokens	RBAC over ACS
Identity Provider Consolidation	Not	Yes

Additional Information

• Each request to the queue must be authenticated. Queues with public anonymous access are not supported. Using SAS, you can build different scenarios for your application.
• The authentication scheme in Azure Queues is to use a symmetric key, which is a SHA-256 hash sum encoded in Base64 . More information on authentication in Azure Queues - Authenticating Access to Your Storage Account , for Service Bus Queues - Shared Access Signature Authentication with Service Bus .
• The Microsoft Azure Active Directory Access Control (also known as Access Control Service or ACS) supported Service Bus provides three different roles: Admin , Sender and Receiver , which are not currently supported by Azure Queues.
• Because the Service Bus supports integration with ACS, you can set up federation with Active Directory (through the use of ADFS) and publicly available identification providers such as Live ID, Google, Facebook or Yahoo.

Prices

Comparison criterion	Azure queues	Service Bus Queues
Transaction cost	$ 0.0005 (per 10,000 transactions)	$ 0.01 (per 10,000 posts)
Paid operations	Everything	Only Send and Receive (the rest are free)
Idle transactions	Are paid (a request for a message with an empty queue costs as much as a request for messages)	Are paid (request for a message from an empty queue is charged as receiving a single message)
Data storage cost	$ 0.07 (per GB per month)	$ 0.00
Cost of external traffic	$ 0.12 - $ 0.19 (depends on the data center)	$ 0.12 - $ 0.19 (depends on the data center)

Additional Information

• Data transfer is charged based on the total amount of data originating from the Azure data center to the Internet for the paid period.
• Data transfer between Azure services within a single data center is free of charge.
• Currently, incoming traffic is not paid.
• The cost of ACS transactions when using the Service Bus Queues is very small. Service Bus Queues generates one ACS token for each queue and uses it until it becomes outdated (about 20 minutes). Thus, the number of transactions is not directly proportional to the number of ACS transactions.
• In situations where the message must be received very quickly, it is more cost-effective to use the Service Bus Queues and the notification mechanism when new messages arrive.

Comment

Prices may vary. Look at the Pricing Overview page.

Conclusion

The decision to choose between Azure Queues and Service Bus Queues depends on many factors, the needs of your application and its architecture. If you’re already using other Azure services, such as tables or blobs, you’ll just start using Azure Queues, especially if you only need basic queue usage scenarios. On the other hand, Service Bus Queues offers a number of additional features, such as sessions, transactions, duplicate detection, automatic processing of “dead” messages, which may be necessary for your application.