Windows Azure: Queues and Service Bus Queues - Similarities and Differences
This article analyzes the similarities and differences between the two types of queues available in the modern Windows Azure platform: Windows Azure queues and Windows Azure service bus queues. Based on this information, you can highlight the similarities and differences of the relevant technologies and choose the solution that best meets your needs.
Introduction
Windows Azure supports two types of queuing mechanisms: Windows Azure queues and service bus queues .
')
The Windows Azure queues that are part of the Windows Azure storage infrastructure support a simple REST interface with Get / Put / Peek functions to create reliable and fault-tolerant messaging between and within services.
Service bus queues are part of a larger Windows Azure messaging infrastructure that supports queuing, as well as publishing, subscribing, web services, and integration patterns.
Both queuing technologies exist in parallel, but Windows Azure queues have appeared earlier. This add-on for Windows Azure storage services is a specialized mechanism for working with data warehouses using queues. Service bus queues are presented in the newest release of the service bus. They are an add-on to a more extensive messaging infrastructure. This infrastructure is designed to integrate applications or application components that can use several different messaging protocols, data contracts, trusted domains, and network environments.
This article provides a comparative analysis of the two queuing technologies implemented in Windows Azure. The modes of operation and functionality of both technologies are discussed. The article also provides recommendations for choosing the functions best suited for solving your application development tasks.
Technology selection
Windows Azure queues and service bus queues are an implementation of Message Queuing service that is currently available in Windows Azure. These technologies have different sets of functions. You can choose either of them or both at once, depending on your needs or the problem being solved (technical or commercial).
When choosing a queuing technology for a specific solution, its architects and developers should consider the recommendations below. For more information, see the following section.
Architects or solution developers need to consider using Windows Azure queues in the following cases.
- The application needs to store in the queue more than 5 GB of messages, and the lifetime of these messages does not exceed seven days.
- The application requires flexible lease options for handling messages. This allows you to assign messages to a very short rental period. Then in the event of a critical failure of the workflow, you can quickly re-process the message. It also allows the workflow to extend the lease of a message if additional time is required to process it. This feature is useful for non-deterministic message processing time.
- The application requires the ability to track the processing of messages that are attached to other messages. This is useful in case of a critical failure of the workflow that processes the message. Then the next workflow can use this information to continue processing the message from the place where the previous workflow ended.
- Server logs are required for all transactions performed on queues.
Architects or solution developers need to consider using Windows Azure service bus queues in the following cases.
- Full integration with the Windows Communication Foundation (WCF) data exchange stack, which is implemented in the .NET Framework, is required.
- Support is needed to automatically detect duplicate messages.
- The ability to process related messages as a separate logical group is required.
- Transaction and atomicity support is required when sending or receiving multiple messages from a queue.
- The lifetime (time-to-live, TTL) for the workload of specific applications may exceed a seven-day period.
- The application processes messages that can exceed 64 KB in length, but most likely will not reach the limit of 256 KB.
- Queue support is required, which ensures guaranteed message delivery in the “first in, first out” mode (first-in-first-out, FIFO).
- Requires the ability to receive messages without polling the queue. When using the service bus, this is a getting operation with a long polling time.
- It is necessary to implement a role-based access model for queues, as well as to provide support for various rights and authorities for senders and recipients.
- The size of the queues used will not exceed 5 GB.
- There are prospects for migration from point-to-point data exchange based on queues to the use of a messaging pattern. This template will provide integrated integration of additional recipients (subscribers), each of whom receives independent copies of some or all of the messages sent to the queue. This is possible thanks to the publish and subscribe feature built into the maintenance bus.
- The messaging solution must support At-Most-Once Delivery Assurance (Maximum Once) without the need to develop additional infrastructure components.
- Requires the ability to publish and process batch messages.
Comparing Windows Azure Queues and Service Bus Queues
The tables in the following sections contain logically grouped queue functions. These tables clearly demonstrate the difference in the capabilities of Windows Azure queues and service bus queues.
Basic features
This section provides a comparative analysis of the main functions of Windows Azure Queues and service bus queues.
| Comparison criterion | Windows Azure Queues | Service Bus Queues |
| Ordering guarantee | Not | Yes - “first entered, first released” (FIFO) (using messaging sessions) |
| Delivery guarantee | At-Least-Once (At least Once) | At-Least-Once (At least Once) At-Most-Once (Maximum Once) |
| Transaction support | Not | Yes (using local transactions) |
| Reaction to receive | Non-blocking (terminates immediately if new messages are not found) | Blocking (with and without waiting time) (support for long polling intervals or method based on long-lived HTTP connections ) Non-blocking (only using the API managed by the .NET platform) |
| Receiving mode | Read and rent | Read and lock Receiving and deleting |
| Exclusive access mode | Based on rent | Lock based |
| Duration of lease and blocking | 30 seconds (default) 7 days (maximum) | 60 seconds (default) 5 minutes (maximum) |
| Detail lease / lock | Message level (each message can have its own timeout value) | Queue level (each queue has its own lock specification that is valid for all messages and fixed for the entire period of the queue’s life) |
| Batch receipt | Yes (explicit indication of the number of messages upon receipt, no more than 32 messages) | Yes (including prefetching implicitly (using a parameter) or explicitly (using transactions)) |
| Batch shipping | Not | Yes (using transactions or batch sending by the client) |
Additional Information
- To use the FIFO guaranteed delivery pattern used in the maintenance bus queues, you must use messaging sessions. In the event of a critical application crash during the processing of a message received in Peek & Lock mode (Read and Lock), when you start the next messaging session, you will receive a message about the failure after a waiting time.
- Service bus queues support At-Least-Once delivery guarantee. In addition, the semantics of the At-Most-Once mode is supported by using session states in which the state of the application is stored, as well as by using transactions for atomizing message retrieval and updating the session state. Windows Azure Workflow Service uses this technique to ensure guaranteed At-Most-Once delivery.
- Service bus queues support local transactions in the context of a single queue.
- Receive and Delete mode supported by the service bus, while reducing delivery guarantees, reduces the number of control messages and reduces the associated costs.
- Windows Azure queues support the lease feature with the option to extend the lease of messages. This allows workflows to provide short message rental times. Due to this, in case of a critical failure of the workflow, another workflow can quickly process the message. In addition, the workflow may extend the lease of a message if its processing period exceeds the current lease time.
- Windows Azure queues support visibility timeout, which can be set when a message is placed on a queue or when it is removed from a queue. In addition, you can update the message by setting new values ​​for the lease parameters in the runtime environment, as well as update various parameters in the messages of the same queue. In contrast, the service bus locks the timeout parameters specified in the queue metadata. These parameters cannot be changed without re-creating the queue.
- The maximum wait time for a blocking receive operation that is available for the maintenance bus is 24 days.
- Client-side batch processing supported by the service bus allows the queue client to batch process several messages in a single send operation. Batch processing is available only for asynchronous send operations.
Extended capabilities
This section provides a comparative analysis of the advanced capabilities of Windows Azure queues and service bus queues.
| Comparison criterion | Windows Azure Queues | Service Bus Queues |
| Scheduled Delivery | Yes | Yes |
| Automatic marking of deleted messages | Not | Yes |
| Deferred messaging | Yes (using update on-site visibility timeout) | Yes (using the specialized API function) |
| Fault reporting support | Yes | Yes |
| In-place upgrade | Yes | Not |
| Server side transaction log | Yes | Not |
| Storage metrics | Yes | Not |
| Queue cleaning function | Yes | Not |
| Message groups | Not | Yes (using messaging sessions) |
| Duplicate message detection | Not | Yes (configured on the sender side) |
| WCF integration | Not | Yes (contains ready-made WCF bindings) |
| WF Integration | Customization (requires the creation of a custom action WF) | Own (contains ready-made WF actions) |
Additional Information
- As of November 2012, both queue processing technologies implemented in the Windows Azure SDK release allow you to create deferred sending schedules.
- Windows Azure Queues provide an update of message content. This feature allows you to save status information and add processing information to a message in order to continue processing the message from the last known checkpoint instead of starting over.
- The tagging mechanism for deleted messages is supported only by service bus queues. It is designed to isolate messages that could not be successfully processed by the receiving application. This mechanism is also used when the message life time (TTL) has expired and the message cannot reach the recipient. The TTL value defines the period during which the message remains in the queue. When using the maintenance bus, the message will be moved to a special queue called $ DeadLetterQueue after its lifetime.
- To detect failure messages in Windows Azure queues, the application analyzes the DequeueCount property of this message when deleting a message from the queue. If the DequeueCount value exceeds the specified threshold, the application moves the message to the message queue to be deleted.
- Windows Azure queues allow detailed logs of all queuing transactions, as well as aggregate indicator logs. These logs are intended for debugging and analyzing the use of Windows Azure queues by an application. They are also useful for optimizing application performance and reducing the costs associated with queuing.
- The “message session” mechanism supported by the service bus allows you to associate messages with a specific logical group with a specific recipient. In turn, this leads to the creation of a connection between messages and recipients, in its properties similar to a session. This extended service bus feature can be enabled by setting the Message SessionID property. Recipients awaiting the transmission of a specific session identifier may receive messages with this identifier.
- Duplicate message detection is supported by the service bus. This function uses the MessageID property and automatically deletes duplicate messages sent to a queue or subject.
Capacities and quotas
This section provides a comparative analysis of Windows Azure Queues and service bus queues in terms of capacity and effective quotas.
| Comparison criterion | Windows Azure Queues | Service Bus Queues |
| Maximum message size | 64 KB (48 KB using Base64 encoding) | 256 KB (including the header and text of the message; the maximum header size is 64 KB) |
| Maximum queue size | 100 TB (limited to one storage account) | 1, 2, 3, 4 or 5 GB (determined when creating a queue) |
| Maximum lifetime of the message | 7 days | With no restrictions |
| Maximum number of queues | With no restrictions | 10,000 (for one service namespace, can be increased) |
| Maximum number of concurrent clients | With no restrictions | With no restrictions (A limit of 100 simultaneous connections applies only to TCP communication) |
Additional Information
- For Windows Azure queues, you must use Base64 encoded messages if their contents are not XML-safe. For a Base64 message, the user payload may be reduced from 64 to 48 KB.
- When using service bus queues, each message stored in the queue consists of two parts: a header and a text field. The total length of the message cannot exceed 256 KB.
- When exchanging data between clients using the service bus queues and the TCP protocol, you can create up to 100 simultaneous connections to one service bus queue. This number is common for senders and receivers. If the quota is exceeded, subsequent requests to create additional connections will be rejected; an exception will be received with the calling code. This quota does not apply to clients that connect to queues using a REST-based API.
- The maximum queue size is enforced by the service bus. The maximum queue size is specified when the queue is created and can be 1, 2, 3, 4, or 5 GB. If the maximum queue size is exceeded, all subsequent incoming messages will be rejected with an exception by call code. For more information about service bus quotas, see the Windows Azure Service Bus Quotas document.
- If you need more than 10,000 queues in the same service namespace associated with a service bus, you can contact the Windows Azure support team and ask it to increase this value. If you need to scale the solution and use more than 10,000 service bus queues, you can create additional service namespaces using the Windows Azure management portal.
Management and operation
This section provides a comparative analysis of the Windows Azure queue management and service bus queue management features.
| Comparison criterion | Windows Azure Queues | Service Bus Queues |
| Control protocol | REST over HTTP / HTTPS | REST over HTTPS |
| Runtime protocol | REST over HTTP / HTTPS | REST over HTTPS TCP with TLS support |
| API running .NET | Yes (Storage Client API Running .NET) | Yes (.NET Intermediate Messaging API) |
| Java API | Yes | Yes |
| PHP API | Yes | Yes |
| Node.js API | Yes | Not |
| Support for custom metadata | Yes | Not |
| Queue Naming Rules | Up to 63 characters (in the queue name only lower case should be used) | Up to 260 characters (queue names are not case sensitive) |
| The function of getting the queue length | Yes (approximate value) | Yes (exact value for a specific point in time) |
| Read function | Yes | Not |
Additional Information
- Windows Azure queues support arbitrary attributes that can be used in the queue description. Each attribute is a name-value pair.
- Windows Azure queues also allow you to read messages without blocking them. This feature is used when developing software tools for viewing queues.
- The intermediate messaging API, implemented for the service bus using the .NET platform, uses full-duplex TCP connections, which improves performance compared to using REST over HTTP.
- Windows Azure queues impose certain restrictions on queue names. For more information, see the Naming Queues and Metadata article (Naming Queues and Metadata) .
- The name of the queue for the maintenance bus cannot exceed 260 characters; for these queues, less stringent naming rules apply. Service bus queue names can contain only letters, numbers, periods (.), Dashes (-), and underscores (_).
Performance
This section provides a comparative analysis of Windows Azure Queues and service bus queues in terms of performance.
| Comparison criterion | Windows Azure | |
| 2000 | 2000 ( 1 ) | |
| 10 ( TCP Nagle ) | 100 | |
| HTTP 503 (, , ) | HTTP 503 (, , ) |
Additional Information
- Windows Azure 2000 . : Put , Get Delete . ( Put ) , , ( Get ), ( Delete ). , .
- Windows Azure, HTTP 503 Server Busy. .
- Windows Azure 10 ( 10 ) , () .
- Windows Azure . , . , , .
- , 2000 1 . . Best Practices for Performance Improvements Using Service Bus Brokered Messaging ( ) .
- When the maximum bus bandwidth of the service bus queue is exceeded, either a ServerBusExException is returned to the queue client (when using the .NET intermediate message exchange API), or an HTTP 503 response (when using the REST-based API). This indicates that the queue is in the mode of controlling the number of requests
Authentication and Authorization
This section provides a comparative analysis of the authentication and authorization features of Windows Azure queues and service bus queues.
| Comparison criterion | Windows Azure Queues | Service Bus Queues |
| Authentication | Symmetric key | ACS Applications |
| Role Based Access Control | Not | Yes (using ACS roles) |
| Federation of Identity Providers | Not | Yes |
Additional Information
- , , . .
- , Windows Azure, , - (Hash-Based Message Authentication Code, HMAC), SHA-256 Base64 . Authenticating Access to Your Storage Account ( ) .
- Windows Azure Access Control (ACS), supported by the service bus, contains three distinct roles: Admin (Admin), Sender (Sender) and Receiver (Recipient), which are not currently supported by Windows Azure queues.
- The service bus supports ACS integration, so you can integrate it with Active Directory (using ADFS) and with frequently used online identity providers, such as Live ID, Google, Facebook and Yahoo.
Expenses
This section provides a comparative analysis of the costs of using Windows Azure queues and service bus queues.
| Comparison criterion | Windows Azure Queues | Service Bus Queues |
| Transaction Cost for Queue | 0,01 . ( 10 000 ) | 0,01 . ( 10 000 ) |
| Everything | ( ) | |
( ) | ( ) | |
| 0,14 . ( ) | 0,00 . | |
| 0,12–0,19 . ( ) | 0,12–0,19 . ( ) | |
| Windows Azure Access Control (ACS) | 0,00 . ( ACS ) | 1,99 . ( 10 000 ; . ) |
Additional Information
- , Windows Azure .
- Windows Azure, , .
- .
- ACS . ACS . , 20 . , ACS , .
- , .
Note.All cost indicators are subject to change. The table above reflects pricing at the time of this writing. The article does not contain promotional offers available at the time of reading. For current pricing information, see the Pricing Overview page .
Conclusion
Obviously, deciding whether to use Windows Azure queues or service bus queues depends on a large number of factors. These factors depend on the specific requirements of your application and its architecture. If your application already uses the basic functions of Windows Azure, you can select Windows Azure queues. This is especially true if you need basic data and message exchange between services, or if you need queues that exceed 5 GB.
Service bus queues have many advanced features: sessions, transactions, duplicate detection, automatic queuing for deletion, fault-tolerant publishing and subscription tools. You can choose this solution if you are creating a hybrid application, or if the above features are needed for your application to work.
At the beginning of this article, an overview of the prescriptions and recommendations is given. Next, we consider the capabilities of queuing technologies available in the Windows Azure environment. The grouping of features by function allows for a visual comparative analysis to identify similarities and differences between Windows Azure queues and service bus queues.
An in-depth study of these issues will help you choose the queuing technology that is most appropriate for specific tasks.
see also
Best Practices for Leveraging Windows Azure Service Bus Brokered Messaging API ( API Windows Azure)
Best Practices for Performance Improvements Using Service Bus Brokered Messaging ( )
How to Use Service Bus Queues ( )
How to Use the Queue Storage Service ( )
Introducing Queues and Topics in Azure Service Bus ( , Azure)
The Developer's Guide to Service Bus ( ; )
Windows Azure Tables and Queues Deep Dive ( Windows Azure — )
Windows Azure Storage Architecture ( Windows Azure)
Using the Queuing Service in Windows Azure ( Windows Azure)
Understanding Windows Azure Storage Billing – Bandwidth, Transactions, and Capacity ( Windows Azure — , )
Source: https://habr.com/ru/post/156715/
All Articles