RabbitMQ

Page last updated:

Integrating the Service With Your App

After the creation of the service and the binding of the service to the application, the environment variable VCAP_SERVICES is created. Information about the credentials are stored in this variable as shown here:

{
  "rabbitmqent": [
    {
    "credentials": {
      "hostname": "rabbitmq.service.consul",
      "hostnames": [
       "rabbitmq-node-0.service.consul",
       "rabbitmq-node-1.service.consul",
       "rabbitmq-node-2.service.consul"
      ],
      "http_api_uri": "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq.service.consul:15672/api",
      "http_api_uris": [
       "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-0.service.consul:15672/api",
       "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-1.service.consul:15672/api",
       "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-2.service.consul:15672/api"
      ],
      "password": "rvUgjRjnMa6mejiW",
      "protocols": {
       "amqp": {
        "host": "rabbitmq.service.consul",
        "hosts": [
         "rabbitmq-node-0.service.consul",
         "rabbitmq-node-1.service.consul",
         "rabbitmq-node-2.service.consul"
        ],
        "password": "rvUgjRjnMa6mejiW",
        "port": 5672,
        "ssl": false,
        "uri": "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264",
        "uris": [
         "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-0.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264",
         "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-1.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264",
         "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-2.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264"
        ],
        "username": "i0zRgICRqzDB5fll",
        "vhost": "bd0c989a-8a16-4641-8bcc-70a99c7c7264"
       },
       "management": {
        "host": "rabbitmq.service.consul",
        "hosts": [
         "rabbitmq-node-0.service.consul",
         "rabbitmq-node-1.service.consul",
         "rabbitmq-node-2.service.consul"
        ],
        "password": "rvUgjRjnMa6mejiW",
        "path": "/api",
        "port": 15672,
        "ssl": false,
        "uri": "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq.service.consul:15672/api",
        "uris": [
         "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-0.service.consul:15672/api",
         "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-1.service.consul:15672/api",
         "http://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-2.service.consul:15672/api"
        ],
        "username": "i0zRgICRqzDB5fll"
       },
       "mqtt": {
        "host": "rabbitmq.service.consul",
        "hosts": [
         "rabbitmq-node-0.service.consul",
         "rabbitmq-node-1.service.consul",
         "rabbitmq-node-2.service.consul"
        ],
        "password": "rvUgjRjnMa6mejiW",
        "port": 1883,
        "ssl": false,
        "uri": "mqtt://bd0c989a-8a16-4641-8bcc-70a99c7c7264%3Ai0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq.service.consul",
        "uris": [
         "mqtt://bd0c989a-8a16-4641-8bcc-70a99c7c7264%3Ai0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-0.service.consul",
         "mqtt://bd0c989a-8a16-4641-8bcc-70a99c7c7264%3Ai0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-1.service.consul",
         "mqtt://bd0c989a-8a16-4641-8bcc-70a99c7c7264%3Ai0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-2.service.consul"
        ],
        "username": "bd0c989a-8a16-4641-8bcc-70a99c7c7264:i0zRgICRqzDB5fll"
       },
       "stomp": {
        "host": "rabbitmq.service.consul",
        "hosts": [
         "rabbitmq.service.consul",
         "rabbitmq-node-0.service.consul",
         "rabbitmq-node-1.service.consul",
         "rabbitmq-node-2.service.consul"
        ],
        "password": "rvUgjRjnMa6mejiW",
        "port": 61613,
        "ssl": false,
        "uri": "stomp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq.service.consul",
        "uris": [
         "stomp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-0.service.consul",
         "stomp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-1.service.consul",
         "stomp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-2.service.consul"
        ],
        "username": "i0zRgICRqzDB5fll",
        "vhost": "bd0c989a-8a16-4641-8bcc-70a99c7c7264"
       }
      },
      "ssl": false,
      "uri": "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264",
      "uris": [
       "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-0.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264",
       "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-1.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264",
       "amqp://i0zRgICRqzDB5fll:rvUgjRjnMa6mejiW@rabbitmq-node-2.service.consul/bd0c989a-8a16-4641-8bcc-70a99c7c7264"
      ],
      "username": "i0zRgICRqzDB5fll",
      "vhost": "bd0c989a-8a16-4641-8bcc-70a99c7c7264"
    },
    "label": "rabbitmqent",
      "name": "testrabbit",
      "plan": "usage",
      "tags": []
    }
  ]
}

Offered protocols and plugins

RabbitMQ provides the following protocols which are all exposed in VCAP_SERVICES:

  • AMQP
  • MQTT
  • STOMP
  • HTTP (API and management interface)

The application should only use the uri or hostname property of the desired protocol within VCAP_SERVICES to achieve proper loadbalancing. These properties contain rabbitmq.service.consul so the connection is established to a HAProxy that automatically distributes requests to all available RabbitMQ nodes.

The alternative properties uris and hostnames should only be used if single nodes need to be targeted explicitly. This can be used to achieve slightly better performance by connecting publishers and subscribers to the queue master node. However, this comes with a serious disadvantage, since the connections fail in case the single node goes down.

The full list of exposed plugins:

  • rabbitmq_mqtt
  • rabbitmq_stomp
  • rabbitmq_management
  • rabbitmq_management_visualizer
  • rabbitmq_consistent_hash_exchange

Advanced configuration

In most cases, connecting your applications to the loadbalanced service is completely sufficient. For some high end setups, getting the last drop of performance and scalability from the cluster is an advanced task and might include strategies which need more logic on the client side:

Non-HA queues

By default, every queue created on RabbitMQ is created on one node (the queue master) and is replicated to other nodes to achieve high availability.

In certain cases, high availability is sacrificed to avoid the overhead of replication to achieve higher performance. For this, just create the queue with the prefix appcloud-non-ha and it will not be replicated to other nodes.

Consistent Hash exchange

For a different approach to scale your queues on the cluster, the Consistent hash exchange plugin is enabled.

Limitations

The RabbitMQ service enforces some limitations to ensure the cluster’s performance.

Per-Queue Message TTL of 14 days

To protect the cluster’s disk from filling up with old messages that were never picked up, the per-queue message TTL is set to 14 days. Messages that are in the queue for longer than 14 days are automatically discarded.

Operator Policies

The RabbitMQ cluster implements operator policies to set certain limits on queues and messages. The following operator policies are defined:

  • max-length: 100’000: If a queue holds 100'000 messages, the most recently published messages will be discarded.
  • max-length-bytes: 100MB: If a queue holds more than 100MB of message data, the oldest messages are discarded from the head of the queue.

Connection Limits

To avoid high load caused by too many open connections, a limit on service instance level is implemeted. A single service instance can hold a maximum of 500 connections. Once this limit is reached, all further connections will be terminated.

Managing your RabbitMQ instances

To get access to the RabbitMQ management console, you can use the cf ssh command. Here’s how it works:

  1. You’ll need to use an app as a host for SSH-ing. Any running app will do. If you don’t have one yet, push a default app through the web console
  2. Create a service key for your RabbitMQ instance with cf create-service-key <service-name> manage (replace the term in <> with your service’s name)
  3. Look at the service key’s credentials with cf service-key <service-name> manage
  4. SSH into your app with cf ssh -L 13000:<credentials.hostname>:<credentials.management_port> <app-name> (replace the values in <> with the actual values from the service key and with your app’s name)
  5. Open your browser and visit http://localhost:13000 and use <credentials.username> and <credentials.password> from your service key as the credentials
  6. When you’re done, simply close the terminal window with the open SSH connection

Sample Application

Cloud Foundry: RabbitMQ Example

Best Practices

 Queues

Keep queues short

RabbitMQ queues work fast and reliably when the they’re empty / have only a few messages queued up. RabbitMQ is not designed to hold message over a long period of time and it should always be the goal to keep the queues empty. Not only does this minimise the risk of message loss, it also increases the overall stability of the cluster and its performance.

Know when to use lazy queues

Lazy queues are queues that save their messages to disk to achieve some persistency. This has a negative impact on the performance of the specific queue but more predictable performance overall when as it minimises memory usage. Small and fast queues should (in general) not be declared as lazy queues. For large queues that don’t require a fast throughput, lazy queues are advised. Lazy queues are enabled by default on the Application Cloud RabbitMQ service offering.

Auto-delete queues

Queues will remain forever on the cluster by default (even when they’re not active). This will put unnecessary strain on the cluster. To free up resources and don’t run into limits, queues should be declared with the auto-delete flag set to true. With this, the queues will be deleted as soon as there are no more active connections using them.

 Connections

General

Connections in RabbitMQ use up a lot of memory. In order to keep the cluster healthy and fast, it is important that there are as few open connections as possible. This can be achieved by keeping connections open for client applications. A good practise is to keep the connection open throughout the lifetime of the client application.

Auto reconnect

Connections between client applications and the RabbitMQ cluster are stable for the most part. However, it’s possible that a network outage can occur or the cluster has an unexpected downtime. In such a case, the client application usually needs to reconnect to the cluster and reopen connections and channels. To automate this process, most RabbitMQ Client APIs support the auto-reconnect feature (see the Java Client API for an example). It is advised to enable the auto-reconnect feature on the client applications.

Channels

General

Channels share the same as restrictions as connections - they can use up quite a lot of memory if opened and closed frequently. Although less memory is used when creating a channel, it’s still recommended that channels are kept open for a long time.

Threads

Channels are not thread safe and should not be shared between different threads.

Prefetch

RabbitMQ allows the for the control over how many messages are sent to the consumer at the same time. Per default, the prefetch value is sent to unlimited, meaning the server tries to send as many messages as possible at the same time. Setting a proper prefetch value can help to achieve the desired performance of the service. If the prefetch count is set too low, the server while idle and wait for more the client to send more message. On the other hand, if the prefetch count is too high the client might take a long time to consume the messages.

Setting the correct prefetch value depends on the setup of the client application. As such, there is not one prefetch value that is best for all cases. As a general rule the prefetch count should be higher if there is only one / few consumers that consume messages quickly. If the client application feature many consumers, a lower prefetch value should be set. More information can be found here: https://www.rabbitmq.com/consumer-prefetch.html.

Messages

Message size

Large messages can take up a lot of resources and should generally be avoided. RabbitMQ is designed for high message throughput and as such it is better to keep the messages sent over RabbitMQ small. Splitting up large payloads into smaller message should be considered when designing the client application.

Message ack’s and confirms

RabbitMQ features acknowledgments (ack’s) for receiving message and publisher confirms (confirms) for sending messages. These mechanisms can be used to ensure that a message has been send / received correctly. Ack’s and confirms are generally a good idea to implement as they increase the reliability and consistency of the message flow.

Note: Ack’s and confirms have a small performance impact.

View the source for this page in GitHub