Day 14 – Project Transit Notifier

To do: Work on breaking my process_transit_request_function to process one item of the queue at a time, read more about threads

What I did: I worked on doing a small sample of Celery~ness. I did an example celery process that was very helpful. My baby example had me pip install Celery & RabbitMQ and create three files:

The example celery code
The example celery code
  • – it creates an instance of Celery and configures it
  • – the configuration details that uses
  • – the actual function for celery to do

To get this baby going, I have to have three terminals open (this is the interface of running programs and talking to the “hardware” part of the computer) and run different parts of my application:

The example celery works!
The example celery works!
  • Terminal #1 – activates my rabbit server, this is where my celery can do its processes
  • Terminal #2 – activate my celery worker, so they can receive the task
  • Terminal #3 – run my python task script, so I can give my worker’s their task

Great! Now to my project….

My celery's -- this is where I have the function (task) I want celery to do
My celery’s — this is where I have the function (task) I want celery to do

So I moved this around and I added to my the information from from my example so I can create an instance of celery. Almost everything that was in is need (expect for the args, I don’t need to pass any arguments), so I am my own file in my directory. Then pulled from my the process_transit_request function and placed it into its own so I can run him independently.

My celery is connected and is processing my queue!
My celery is connected and is processing my queue!

To get this connected, Kai (an instructor) let me explain to her what I was doing and the steps I need to do to get it started. She was so helpful and was able to the questions and help me figure out that I put put part of my configuration instruction in my when it should be in my We were able to get an error from celery! YEAH! I never thought that I would be so happy to see an error. This means that I am able to go with celery into my task and get him going. Its currently not able to process the request to get a list of transit request that need to be process, but I can figure that one, because now that i just the python file getting access to my database. Thank you so much Kai for being supportive!

To do Tomorrow: Fine tune my celery processes and add “processes_line_and_bound_selects_closest_vehicle” function to task so my user doesn’t have to wait for it to be processed

My Thoughts/Process: These past few days working on understanding celery, rabbit, threads, locks, deadlocks, etc. was challenging – there were many times that I felt I was starting to understand and then it would slip. It really wasn’t until I did the baby example did things start solidifying. Then using the white board to organize the set up and work flow gave me a map and then it was a matter of adding it into my code. Once I was there, it was a manner of having someone else (Kai) to explain what I was doing which supported my understanding and logic. The moment I got a celery error I was so excited! I told Kai that I never thought that I would have ever been so happy to see an error — the error meant it connected! Today was very satisfying and I am so happy that I didn’t give up on learning this new technology.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s