Docker Image Building – Lessons learned

My cps-WhatsNew script has a new permanent page on the blog site now, as well as a new Docker image!  Check out the links and let me know what you think.

While the cps-WhatsNew script is relatively simple, I was thinking about how could I make it easier to implement the solution.  Docker was a clear choice.  As I’ve only been a consumer of Docker images, this “short” project was a very interesting and took an, embarrassingly enough, long time to complete.

It took close to 3 full days, off and on, to get everything put together and optimized.  The optimization part was the longest and I’m still not happy with the size of the docker image now that I’ve published everything.

Here are some hard-won lessons:

Read moreDocker Image Building – Lessons learned

Updated cps-WhatsNew

After months of procrastination and over-engineering, I’ve finally made some updates to the cps-WhatsNew program.

New features:

  • Dev mode – allows for the testing of the script by changing a setting in the configuration file.  The program will do everything normally except it will not send any emails.
  • Database mode – added the ability to change where the program gets it data for the distribution list.  It can now be either the configuration file or the cps application database.
  • DL Exclusion list – added the ability to keep an exclusion list of emails.  This is really only used with the Database mode above.  Basically there is a list of email addresses in the configuration file that can be edited.

I set up this program on my server that contains the Calibre-Web application.  As the database is a SQLite database, its pretty important that we don’t write to it at the same time that the cps application is running. However, I’ve scheduled the timing of sending of this newsletter at an off-peak time so everything should be okay.

That last bit is where I was over engineering everything.  I had originally thought to create another database entirely for the cps-WhatsNew program to avoid any sort of contention with the Calibre-Web database.  I thought I would simply sync the user table over – or really just a subset of the data and add a column for people that wanted to be excluded from the newsletter.  After starting down this path and creating a lot of the CRUD code, I realize that I was really over-engineering the solution.  I could simply just do a select statement on the existing database to accomplish everything I needed.  The database contention would be there with my over-engineered sync’ing solution or the simpler select statement.

I think the only other thing I want to do with the program is make it easier to customize the introduction paragraph of the newsletter.  Currently everything is hardcoded in the HTML files.  I’ll probably just have the program look for a specific file for the introductory paragraph.  Thus seperateing the message from the main body of the newsletter. This should be pretty easy to add.


cps-WhatsNew – Newsletter for Calibre-web

Thought I would do a quick post on a new git repo I just put together:  cps-WhatsNew.

Its Python script that puts together a newsletter for the calibre-web repo.  I love the calibre-web repo as an interface to my calibre instance.  For my purposes, I thought it would be cool to send an email newsletter letting users know what new books have been posted.

While I haven’t tested it, I’m sure it would work for most OPDS feeds.  — Maybe with some minor adjustments to the book thumbnail code.

In any event, this was my first official Python script.  I’m sure its over-commentated and over-engineered – just like everything else I do!   I think it turn out well.  I used/learned, besides general Python, templating with jinja2 (without flask), a great emailing package – marrow, in-depth logging.  Also starting using JetBrains PyCharm IDE – Community Edition.  Excellent IDE of which i’m certain I would never use it to its fullest. (good GIT integration!)

I’m pretty impressed with Python.  I can’t believe it’s taken me this long to take the time to start learning the language.

AirPort Express and AT&T U-Verse Pace 5268AC

Very interesting bug I thought I’d document somewhere.  My current home ISP service is AT&T UVerse and I recently Pac 5268AChad it serviced because of some issues.  I ended getting a whole new router – the Pace 5268AC. This was very exciting because all of my other equipment is no faster the N speed — this router gives me wireless AC.  Woot!  Although the router is so much better than what I had, still had some issues with my AirPort Express.

I had setup two v1 AirPort Express devices to some powered speakers. One that I kept outside.  They stop working well over a year ago and I thought it was because of a bad connection to the older router.  Still had the bad connection issue after the new router. So it must be the AirPort Express.  So I figure time for another upgrade.  I jumped on eBay and picked up a used late model AirPort Expess.

It arrived in the mail today and I of course couldn’t wait to set it up.  This is where things get interesting.

If you use the Airport Utility application to configure the Airport Express, you will not be given the option to configure the AirPort Expressdevice as a wireless client on the existing wireless network.  You must first set it up in a new network.  Obviously it is much easier to configure the device if you plug in an Ethernet cable to the Airport Express.  Once the device is back up, you can then configure it to be a wireless client on the existing network.  So I basically named the device, entered my wireless name, etc.

Once the AirPort Express restarted – my computer and most of my network couldn’t not access the Internet.  Somehow, some way, the AirPort was seriously screwing up the routing on the network.  I could access anything that was on the LAN side by using IP addresses.  As I use an external DNS server for DNS, I couldn’t access anything by name.  I also couldn’t access the router’s LAN port to see what issue the router would be reporting.  

In order to get everything back online, I had to unplug theater net cable from the AirPort Express and break the connection between my router and the down stream switch the AirPort was plugged into.  Once I reconnected the switch, we were back online.  

Fortunately, the AirPort Express was still connected. I moved it outside and hooked it up to my speakers and it works like a champ!  It’s going to be a nice relaxing weekend with poolside music.