PubSubX
A small publish-subscribe system, on PyPI
- Year
- 2024
- Kind
- Systems
- Status
- Finished
A publish-subscribe broker and client in under 250 lines of standard-library Python, on PyPI as pubsubx. Clients subscribe to topics over TCP, and each message goes to every subscriber of its topic.
PubSubX is a minimal message broker: start the server as a module or from code, then publish and subscribe to topics from any number of clients. It's meant as a building block for small distributed apps that don't need a full broker.
The problem
Small distributed apps often need one thing from a message broker: send a message to a named topic, and have every process listening on it receive it. PubSubX does that and nothing more. It is a broker and a client in under 250 lines of standard-library Python, installable with pip install pubsubx.
How it works
- 01
Start the broker
The server runs as the pubsubx command, as python -m pubsubx or from code, on localhost:5000 by default. It listens on TCP and gives each client its own thread.
- 02
Subscribe to a topic
A client sends JSON such as {"action": "subscribe", "topic": "orders"}. The broker keeps two maps: each topic to its subscribed sockets, and each socket to its topics. The second makes a disconnect cheap: only that client's topics are visited, and topics left empty are deleted.
- 03
Publish a message
A publish message carries the topic and the content. The broker looks the topic up, and if nobody subscribes, the message is dropped. Nothing is stored or acknowledged.
- 04
Fan out a copy per subscriber
The broker sends a copy to every subscriber of the topic except the sender. It takes a snapshot of the subscribers under a lock and sends outside it, then removes any client whose send failed, so one dead subscriber can't take the publisher down with it.
- 05
Deliver to a callback
Every message is one line of JSON. On the client, a background thread reads from the socket into a buffer and splits it on newlines, so a read can hold part of a message or several at once. Each complete message goes to the on_message callback as a topic and its content.
Results
- 3
- releases on PyPI, 0.1.0 to 0.1.2
- 0
- runtime dependencies
- 4.8 KB
- wheel size of the current release
Decisions
Standard library only
It imports only socket, threading and json, so pip install pubsubx pulls in nothing else. The package declares support for Python 3.6 and up.
Don't echo to the sender
A client that publishes and listens on the same topic doesn't get its own messages back. It's one deliberate check in the publish loop.
Timeouts instead of blocking calls
The server's accept and the client's recv both time out every second, so their loops can check a running flag and stop.