hashbang.ca

After working with Yanick Champoux on a few little Perl projects here and there, we finally met face-to-face at YAPC::NA last summer. A few months later, when I was looking for a co-op position, I immediately thought of Pythian.

They’re a mid-sized company in Ottawa that does database services. They’re the expert DBAs you hire if you don’t have your own (or enough of your own). Their in-house development team works on 2 major pieces of software: Support Track, a ticket tracking web application, and Avail, a server monitoring tool. I was hired to do quality assurance on these two systems over the winter semester.

A lot of the testing was manual, and I learned a bunch of new techniques for testing and debugging web applications. Because I did a bunch of little things while I was there, I wanted to share a bunch of little tips & tricks you might find useful if you don’t already know about them.

Testing/debugging tips & tricks

• openssl s_server and s_client are a simple server and client included with the standard OpenSSL installation. If you need to see what your SSL server or client is doing, these provide the instrumentation you’re looking for. I used this extensively in my testing for a project adding SSL support to an HTTP client library – see also SSL Security in HTTP::Tiny.
• telnet, netcat make a nice server/client pair for plaintext protocols like HTTP or memcached.
• mtr is the best of ping and traceroute rolled into a single package.
• perldoc -l will tell you which module is being loaded. Module::Versions::Report is also very helpful.
• Symlinks are often used to provide a kind of versioning for a directory. To switch the symlink to the new target atomically, do: ln -s new-thing link-tmp && mv -Tf link-tmp link – the mv makes is atomic.. If you try to give the symlink a new target with ln directly, it isn’t atomic. Use strace if you don’t believe me.
• cp has a -s flag that makes symbolic links with a less confusing syntax than ln.
• To follow a file being appended to while also being able to scroll back and forth, use less and press F to follow just like tail -f.
• To make columnar data from various commands actually readable, pipe into column -t.
• perltidy is your friend.
• Just do echo 'alias fail="tail -f"' >> ~/.bash_alises already and be done with it!

I was asked to add SSL support to a client library, while also moving from home-grown manual HTTP code to a proper module. HTTP::Tiny was ideal because it is pure-Perl, a core module since 5.14 (so it’ll be maintained), and it’s just one .pm file, making it easy to ship.

An application server that supported SSL was provided for testing purposes, but the SSL certificate didn’t match the hostname – HTTP::Tiny correctly rejected connections. I needed to be able to control the settings sent to the underlying IO::Socket::SSL object used for the encrypted connection so I could turn off security features for testing. As I worked on that, David Golden offered invaluable feedback, which greatly improved the design of the features added to HTTP::Tiny.

As of 0.018, HTTP::Tiny is more configurable, and has a simple interface for easily making SSL connections more secure.

By default, HTTP::Tiny can’t do SSL at all, but if you install IO::Socket::SSL, it can. Unfortunately, even with that, the SSL connection will be insecure – no checking is done that the SSL certificate’s CN matches the hostname of the server we’re connecting to, nor is it checked for validity according to a Certificate Authority. But now, you can add verify_ssl => 1 to the constructor to get more secure operation. The hostname will be validated, and we’ll try to find a CA bundle to verify the server’s certificate with. It is recommended to install Mozilla::CA, which provides a CA bundle, but we’ll try to find the file your operating system provides if possible. If none are found, it is a fatal error, so installing Mozilla::CA is probably a good idea.

If you need finer control (SSL client certs, verifying the hostname but not verifying against a CA bundle, providing your own validation callback), you can pass SSL_options in the HTTP::Tiny constructor, and those SSL_* options will get passed to IO::Socket::SSL::start_SSL. It’s not recommended to do that unless you know what you’re doing – and that’s harder than you think. The documentation is atrocious, but I think we’ve done a good job of hiding the best and most common security settings behind verify_ssl => 1.

TLDR: If you want SSL using HTTP::Tiny to be secure, install HTTP::Tiny version 0.018, IO::Socket::SSL and Mozilla::CA; and use verify_ssl => 1 in HTTP::Tiny’s constructor.

Lessons learned

This episode has made me wonder why Perl doesn’t provide SSL support out-of-the-box. You need an add-on to make it work – and not just any add-on. IO::Socket::SSL is actually not trivial to deploy. You need OpenSSL libraries installed, which requires a compiler. There are systems where that is the end of the road. Maybe you don’t support them, but someone does. Unfortunately, the servers running 5.004000 probably aren’t updating to 5.020000 any time soon, having SSL available in the core install would be a good first step for Perl’s next decade.

Init scripts are annoying little things – almost entirely boilerplate. Here’s how I learned to stop struggling, and love Daemon::Control to control my daemons.

The module really is as simple as the synopsis - you describe the daemon, have it write an init script (which actually just runs your Daemon::Control script) for you, then update-rc.d and you’re golden. It really is that simple. Read more »

I’ve always favoured pastebins that let you bin a paste and nothing more – p.defau.lt and sprunge.us spring to mind. I’ve made a Perl almost-clone of sprunge.us:

http://p.hashbang.ca now runs WWW::Hashbang::Pastebin, a simple pastebin written with Dancer and DBIx::Class that does nothing but store your text and show it back to you. The only feature beyond that is if you append a +, you’ll get line numbering (no syntax highlighting). You can use an anchor to jump to any line (click the line number), and the number for that line will be highlighted.

To interact with the pastebin, just POST with paste content in p and get the URL back in the X-Pastebin-URL HTTP header (and in the body, so curl-ing will Just Work):

Or, use the Perl client, which provides a command-line tool to do the same thing (and also fetch paste content, given an ID).

My pluginbundle for Dist::Zilla includes, among other things, the Twitter plugin so I can brag on Twitter every time I release a module. Mysteriously, it broke one day. Looking at the code, I realized that it was sending my username and password to authenticate. I remember being uneasy with that at the time I set it up, but I quickly forgot, and continued along blithely. OAuth is an authorization standard that allows users to avoid giving their username & password to a potentially-untrusted application. Using OAuth has been requested since 2011, and the 401 Unauthorized error I got indicates that now it is mandatory for Twitter.

David Golden offered to let me maintain the module, and I’m a sucker, so… :D Read more »

Today, I discovered yet again one of the cardinal sins of user interface design – inconsistency. A blog had two pagination interfaces on one page – and they had opposite directionality. For one, older posts were to the left; for the other, older posts were to the right. This is an obvious and egregious error, and the solution is to make them operate in the same direction, or better yet, remove one. Read more »

Apparently, when your Android device’s busybox installation gets broken for whatever reason, this doesn’t cause your phone to come to a screeching, grinding halt. The breakage is much more subtle. For me, the only symptoms were:

• the standard busybox installers available on Android market couldn’t install busybox (this may not be related, who knows!)
• wifi stopped working – the phone couldn’t connect to the AP

Read more »

James Gannon points out that “Critics of the TPM provisions in Bill C-11 often claim to have a “balanced” solution for TPM protection: to create an exception that allows hacking for legal purposes.” That’s certainly correct, however, he proceeds to misconstrue what that suggestion actually means Read more »

Minister Vic Towes responded to Privacy Commissioner Jen Stoddart’s open letter Friday:

Toews was quick to shoot down Stoddart’s concerns in an ongoing battle that pits the government against consumer advocates and privacy experts. “Our approach strikes an appropriate balance between the investigative powers used to protect public safety and the necessity to safeguard the privacy of Canadians,” Toews said in a statement Thursday in response to Stoddart’s letter.

“As technology evolves, many criminal activities – such as the distribution of child pornography – become much easier. We are proposing measures to bring our laws into the 21st Century and provide police with the tools they need to do their job.”

Read more »

Between April 2006 and March 2011, Canada was governed by a minority Conservative government, meaning the government needed the co-operation of the opposition parties to pass legislation. Despite a lot of talk about minority governments necessarily yielding instability, having a minority government forestalled the worst of the Conservatives’ plans. In the last federal election in March, the Conservative Party won a majority of the seats in Parliament – meaning they have enough votes to pass any legislation they want, barring opposition within their own party. Given Prime Minister Stephen Harper’s iron-fisted control over his caucus, that’s unlikely. Read more »

chromatic mentioned how to use dependency injection in You’re Already Using Dependency Injection. Although I had read that when he posted it, I hadnt actually ever done it. That is, until today. Read more »

Earlier this year, I posted a short entry about having accepted a job offer from Defence Research & Development Canada (DRDC). Over the past four months, I’ve had a great supervisor, a relaxed workplace, and challenging work. As you might imagine, working at a defence research lab is quite different from anything I’d done previously. I knew that was going to be the case, but I was still surprised at how little of my prior knowledge applied to The Real World Of Real Work. But first, what awesome, classified, doomsday devices did I get to work on? Well… Read more »

With gedit plugins, you can turn this simple text editor into a lightweight IDE. It’s fast, has good syntax highlighting, and can have code completion, shell integration, and many similar feature you might expect from an IDE. One feature it lacked was trimming whitespace from files. I searched for plugins to do this, and found several, but none of them quite met my expectations, because none were configurable. I typically want my files to end with one and only one newline. Of course, the solution is Perl. Read more »

At YAPC::NA 2011, I whined about the lack of codification of tribal knowledge in Perl. One area that’s ripe for fixing is the documentation on open. There’s a section in perlfunc for open, and a tutorial: perlopentut. That tutorial is where I’ve started my campaign to have the tutorials and FAQs give good advice. Read more »

YAPC::NA 2011 was great (read about it: 1, 2, 3), and I enjoyed almost every minute. I have only a few gripes, collected here in no particular order:

• You can never have enough power in the conference halls. Never.
• Yes, a four-day conference would be great. Give it a try!
• Make the area immediately outside the conference halls a comfortable place to sit and/or mingle. Having an area for hacking at the other end of the building isn’t good enough, it needs to be right there. And close the doors so the talking doesn’t distract attendees listening to talks (or the speakers).
• The auction is a problem. It should absolutely not interrupt prime socializing/networking time. Make it a silent auction for the run-of-the-mill items, limit the live part to a half-hour of the big-ticket items, and don’t be so desperate for large donations – it’s unseemly, exclusionary, and probably raises less money than many smaller donations.

I heard about these from other attendees as well.
However, I want to thank the organizers and volunteers for putting together a really really good conference. I was impressed with how smoothly things went, and I all the attendees I spoke to were highly complementary of the event as a whole.