PgQue Postgres Queue: 1-5μs Latency on Postgres 14+

Nikolay Samokhvalov, software engineer and Postgres contributor, released the PgQue Postgres queue on GitHub on April 8, 2024. The pure PL/pgSQL implementation targets Postgres 14 and later. It achieves 1-5 microsecond latency for send, receive, and ack operations, per Samokhvalov benchmarks. Users deploy it on AWS RDS with the pg_cron extension.

Samokhvalov based PgQue on Skype's PgQ architecture from 2007. PgQ supported messaging for hundreds of millions of users over a decade on self-managed Postgres clusters, per historical engineering reports.

The pg_cron extension schedules pgque.ticker() every 1 second. End-to-end message delivery measures 1-2 seconds in default setups. See the PgQue GitHub repository.

PgQue PL/pgSQL Core Functions

PgQue implements queuing logic in PL/pgSQL. It requires Postgres 14+ and pg_cron for scheduling.

Nikolay Samokhvalov states in the repository README that developers invoke pgque.ticker() at 1-second intervals. This configuration supports high throughput on managed and self-hosted Postgres.

No external brokers like Redis or RabbitMQ required. All operations run inside Postgres.

Latency Benchmarks from PgQue Repository

Per-call latencies measure 1-5 microseconds, per benchmarks by Nikolay Samokhvalov in the PgQue GitHub repository on April 8, 2024.

End-to-end delivery takes 1-2 seconds with the 1-second ticker. Brandur Leach, former Heroku infrastructure engineer, documented a 60,000-job backlog in one hour on Heroku Postgres queues in 2015.

Postgres performs all computations internally.

• Metric: Per-Call Latency · PgQue Value: 1-5 μs · Source Attribution: Nikolay Samokhvalov, GitHub 4/8/24 · Comparison Example: Redis: 100-500 μs, AWS docs
• Metric: End-to-End Delivery · PgQue Value: 1-2 seconds · Source Attribution: PgQue README · Comparison Example: Heroku: 60,000 jobs/hr, 2015
• Metric: Ticker Interval · PgQue Value: 1 second · Source Attribution: pg_cron docs · Comparison Example: N/A
• Metric: RDS Instance Cost · PgQue Value: $0.016/hr USD · Source Attribution: AWS pricing, 4/9/24 · Comparison Example: ElastiCache Redis: $0.025/hr USD

PostgreSQL 14 documentation.

AWS RDS Costs for PgQue Deployments

AWS RDS for PostgreSQL includes pg_cron as a standard extension. This allows automatic ticker scheduling without custom jobs.

A t4g.micro RDS instance costs $0.016 per hour USD, per AWS pricing on April 9, 2024. ElastiCache Redis t4g.micro costs $0.025 per hour USD.

Queues support real-time processing in financial applications. PgQue operates without external brokers.

Self-managed Postgres 14+ instances support PgQue. Skype's PgQ processed billions of messages daily, per period engineering reports.

pg_cron GitHub. AWS RDS for PostgreSQL.

PgQue Compared to Redis and RabbitMQ

Redis offers sub-millisecond latency on separate clusters. RabbitMQ handles complex routing with 10-50ms overhead, per benchmarks.

Nikolay Samokhvalov states PgQue requires no additional infrastructure, lowering costs for Postgres-based applications.

Postgres achieves over 1 million TPS on optimized hardware, per official benchmarks. This supports high-volume workloads.

Brandur Leach noted in 2015 that Postgres queues avoid vendor lock-in. Heroku issues resulted from configuration limits.

Developer Guide for PgQue

Call pgque.send(), pgque.receive(), and pgque.ack() in SQL. PL/pgSQL provides native performance.

Set ticker to 500ms intervals for sub-second delivery. pg_cron supports second-level granularity.

Test on local Postgres 14+: CREATE EXTENSION pg_cron; Invoke pgque.ticker().

Integrate with pg_trgm for queue search or TimescaleDB for time-series data.

PgQue Future Developments

Nikolay Samokhvalov plans Postgres 17 compatibility, per GitHub repository.

Cloud costs rose 15% year-over-year, per Gartner Inc. 2024 forecast. PgQue supports cost-efficient queuing.

PgQue Postgres queue enables low-latency workloads in Postgres.