What is Pub/Sub?

Pub/Sub is an asynchronous messaging service that decouples services that produce events from services that process events. You can use Pub/Sub as messaging-oriented middleware or event ingestion and delivery for streaming analytics pipelines. Pub/Sub offers durable message storage and real-time message delivery with high availability and consistent performance at scale. Pub/Sub servers run in all Google Cloud regions around the world.

Core concepts

• Topic: A named resource to which messages are sent by publishers.
• Subscription: A named resource representing the stream of messages from a single, specific topic, to be delivered to the subscribing application.
• Message: The combination of data and (optional) attributes that a publisher sends to a topic and is eventually delivered to subscribers.
• Message attribute: A key-value pair that a publisher can define for a message. For example, key iana.org/language_tag and value en could be added to messages to mark them as readable by an English-speaking subscriber.

Celery is a task queue with batteries included. It’s easy to use so that you can get started without learning the full complexities of the problem it solves. It’s designed around best practices so that your product can scale and integrate with other languages, and it comes with the tools and support you need to run such a system in production.

pip install celery

Choosing a Broker

Celery requires a solution to send and receive messages; usually this comes in the form of a separate service called a message broker. In this example, I use RabbitMQ on docker for the most simple experiment.

docker run -d -p 5672:5672 rabbitmq

Getting started

SQLAlchemy is most famous for its object-relational mapper (ORM), an optional component that provides the data mapper pattern, where classes can be mapped to the database in open ended, multiple ways - allowing the object model and database schema to develop in a cleanly decoupled way from the beginning.

pip install SQLAlchemy

1. Generate new RSA SSH keys

ssh-keygen -t rsa -b 2048 -C "email@example.com"

for example, I have 2 keys in my laptop

~/.ssh/id_rsa
~/.ssh/tpp_rsa

Next, add the new key you just created

ssh-add ~/.ssh/tpp_rsa

You can check your added keys

ssh-add -l

2. Modify the ssh config

nano ~/.ssh/config

TF-IDF short for term frequency – inverse document frequency, is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus. It is often used as a weighting factor in searches of information retrieval, text mining, and user modeling. The tf–idf value increases proportionally to the number of times a word appears in the document and is offset by the number of documents in the corpus that contain the word, which helps to adjust for the fact that some words appear more frequently in general. tf–idf is one of the most popular term-weighting schemes today.

1. Bag of words

Tokenize a sentence

from nltk.tokenize import TreebankWordTokenizer

sentence = "The faster Harry got to the store, the faster Harry, the faster, would get home."
tokenizer = TreebankWordTokenizer()
token_sequence = tokenizer.tokenize(sentence.lower())
print(token_sequence)