Anagrams – in O(N)

If you understand “big O” notation, skip on down to the “Solving Week 5 Problem 2” heading.

I’m going to discuss this using Perl 6, but I think an intermediate Perl 5 user will be able to figure out what the code does and how it would be able to be implemented in Perl 5.

“Big O” Notation

Most students of computer science have been exposed to “big O” notation.  Essentially, algorithms can be classified by performance.  N represents the size of the data being processed, while O() is shorthand for “execution time on the order of …”

Continue reading “Anagrams – in O(N)”

Open Up Your Open Source Module – Part 4: Contributor Credit

This is Part 4 of a 5 part series.  Links to other parts (as they become available) are below.

We’ve previously talked about adding the CONTRIBUTING, TODO and CODE_OF_CONDUCT files. But how about giving contributors credit? This is a short, and easy, tip.

Continue reading “Open Up Your Open Source Module – Part 4: Contributor Credit”

Open Up Your Open Source Module – Part 3: CODE_OF_CONDUCT File

This is Part 3 of a 5 part series.  Links to other parts (as they become available) are below.

We’ve previously talked about adding the CONTRIBUTING and TODO files, so now we’ll get to the post I’ve been dreading: the CODE_OF_CONDUCT file. I believe strongly in the value of codes of conduct, and think every project should have one.

So, why would I dread it?

Continue reading “Open Up Your Open Source Module – Part 3: CODE_OF_CONDUCT File”

Open Up Your Open Source Module – Part 2: TODO File

This is Part 2 of a 5 part series.  Links to other parts (as they become available) are below.

So, you’ve added a CONTRIBUTING file to your module (in part 1).  That’s great! Now people know the ground rules for contributing to your module.  But there is another thing you can do to encourage the contributions you want: Simply tell the potential developer what changes you might appreciate.

Continue reading “Open Up Your Open Source Module – Part 2: TODO File”

Open Up Your Open Source Module – Part 1: CONTRIBUTING File

You’ve written something awesome! You want to release it as open source software into the world.  And, you hope other people will want to polish the rough edges, oil the squeaking hinge, and translate the help text into Klingon. Assuming that your module is interesting to people, how do you help them help you? I’ll give examples of how to do this in the context of Perl 5 modules, but they’ll work in other languages, too.

I talked about this in a lightning talk @ The Perl Conference 2018

But I also thought I could expand and organize my thoughts a bit better, so for the textually inclined, read on!

This is part one of a five part series.  I’ll link them all here when they are up.

Continue reading “Open Up Your Open Source Module – Part 1: CONTRIBUTING File”

A Minimal Raspberry Pi 3 GPS Time Server

A Raspberry Pi can make an outstanding NTP time server for your network. Below, I document the bare minimum needed to get a Raspberry Pi 3 Model B with the Adafruit Ultimate GPS Hat working as an NTP server.  You can do this, even buying all the parts new, for under $100 USD.

What makes this project interesting is that you end up with potentially a very compact time server – a Pi with a nicely stacked board on top. Just bring network and power to the box, put the box near a window, and it’s otherwise self-contained. For an even simpler deployment, in a corporate environment with PoE switches, use a PoE to USB cable like this one (disclaimer: this is an Amazon affiliate link, so I get a tiny amount of money if you buy something here): PoE to USB Splitter – then all you need is a network cable (no wall wart is required) which supplies connectivity and power.  Note that the PoE (Power over Ethernet) adapter requires that you have ethernet switches that support PoE.  Your business might have these, but you probably don’t have them at home (they cost a lot more money).

The only thing I’m missing is a good case – if anyone knows of a good Raspberry Pi 3 case that fits well with the Adafruit hat installed, I’m very interested — please comment!

Why do you want to do this?  There are several reasons:

  • You have a need for “correct” time on your servers (to do log correlation or for regulatory requirements)
  • You have an asymmetric internet connection.  There is no way for internet time servers to be more accurate than 10ms or so if you have this type of connection – virtually any DSL or cable internet connection.  On these connections, you have a faster download than upload.  That means that that it takes several ms (milliseconds) less time to travel from the internet to your machines than it takes to travel from your machines to the internet.  Because NTP measures round trip delay and divides it by two, assuming equal delay on both paths, your time will be off in such cases. Whether it’s off enough to bother you is a different question!
  • Related to the above, it is very unusual for internet backbone paths to be the same for both directions of your connection.  In fact, this is more common than not. This causes issues with NTP.
  • During network congestion (whether it’s a peering point, a congested NTP server, or your internet connection), you will see additional delay added, typically more in one direction than another.  Right now, NIST’s Maryland servers (time-a.nist.gov, time-b.nist.gov, time-c.nist.gov) have about 70ms of extra delay due to congestion at their end – this makes the time about 35ms off (70 divided by 2) for anyone synchronizing to their servers.
  • You need accurate time on a network not connected to the internet and/or during internet outages
  • You don’t want to be dependent upon volunteer server operators for the correct time
  • You might just like knowing your time is more correct than your friend’s!

There are lots of tutorials on the building NTP servers from Pis, but this one is a bit different because:

  • It’s based on the Pi 3 Model B instead of the 2 or older.  There are some serial oddities on the Pi 3, so you do need to do minor modifications.
  • Most tutorials connect to gpsd via shared memory to fetch the time of day while using the kernel PPS module to get the seconds to start at the right instant.  This means you can have time that is several hundred milliseconds off reality upon startup of NTP.  This can be mitigated slightly by fudging the stratum of the GPS and having internet time services also configured, but it can’t be eliminated completely.
  • I don’t generally manage computers from the console, I use network-based access to do so.  Most tutorials assume it’s convenient to haul a monitor out to the Pi.  This tutorial never requires you to connect a monitor (but you can if you want).
  • This tutorial uses versions of ntpd and gpsd I compiled from more recent sources than come with the stable branch of Raspbian.  The ntp code also has a patch I’ve made to allow it to see both the GPS and PPS devices from gpsd easily.

So, here’s the procedure: Continue reading “A Minimal Raspberry Pi 3 GPS Time Server”

Secrets to Good Home Wifi

You open up your laptop, enter your password, and then start your browser. Maybe you click on a movie.

What happens next? For far too many homes, what happens next is your browser starts playing and then rebuffers the movie. Maybe you get kicked off your wifi completely. The ISP gets blamed – and may be at fault, but probably not for the reason you are yelling at them. The problem is most likely not your internet connection. It’s your wireless network. These tips are for experienced IT people who are comfortable reading IT system manuals, but who don’t have strong expertise in wireless 802.11 networking. Wifi is, sadly, not a “plug-and-play” technology – certainly anyone can learn about it and become an expert, but sometimes it’s better to hire an expert. If you need your Wifi to work like your home’s electricity, and always be there when you need it, sometimes you’ll need to hire experts in the field. But I’m going to assume that you are more of a do-it-yourself type of person who is comfortable learning more about their technology, and are able to read and understand your network devices’ manuals. With that in mind, how do we fix this?
Continue reading “Secrets to Good Home Wifi”

Fixing OSX’s Broken External Display Support

I have two external monitors on my iMac – both connected via mini-DP to DP cables. I didn’t want to spend the money Apple charges for displays, so these are third-party displays with specs very similar to Apple’s displays.

But…the text on them looks like crap. It’s jagged, blocky, and just generally ugly – why? The iMac built-in monitor displays text beautifully.

It turns out that if I go to About This Mac, then to System Report, then to Graphics/Displays, I can see the problem: OSX thinks my two external monitors are televisions. This has color space issues and causes anti-aliasing of text to be disabled. In other words, the display noticeably looks like crap.

I suspect this is a way for Apple to discourage use of third party hardware – I’d like to believe it isn’t, but since my displays have a native resolution unlike that of any TV, it would be simple for Apple to differentiate between TV and monitor. Even better, they could provide an easy interface to override this guess. Besides, Windows and Linux look beautiful on these monitors without any tweaking, so unless anyone thinks Apple lacks the brilliant minds of Microsoft, this isn’t an impossible problem to solve – particularly because Apple has had many years to fix it.

Back to the topic, how do you fix it? Fortunately, others have figured out how to deal with this problem – rather than repeating the solution here, just go to the I Reckon blog and read Force RGB mode in Mac OS X to fix the picture quality of an external monitor .

Making PAR::Packer Work with RPM

I needed to distribute some RPMs with a PAR::Packer created (using pp) executable.

If you don’t know what PAR::Packer is, it is the incredibly awesome Perl 5 utility that allows you to create an executable that contains your Perl interpreter, your code, and any libraries you need.  It’s basically one of the ways you have to escape dependency hell with Perl – you can use all the wonderful CPAN modules without fear. You can use the new features of recent versions of Perl (such as subroutine signatures and enhanced performance), without worrying about what version of Perl is installed on someone else’s machine. You don’t need to worry about missing core modules in Fedora and other platforms (Fedora is missing autodie – an essential module for lazy programmers).  So building an archived executable is a great way to distribute Perl apps to people not comfortable with the Perl toolchain.

You can think of it like a Perl compiler (it’s not, but the distinction isn’t terribly important for this context).

It’s a somewhat finicky beast, and can take some effort to get it to package things the way you need them, but it’s a standard trick of the trade when I can’t just use perlbrew to build what I want on a machine. It certainly is a lot quicker to install a single executable than to compile and test hundreds of CPAN modules!

So, it seems natural to combine this with RPM. This way, users can install my stuff just by doing a simple rpm --install <file>.rpm – that’s awesome.  Well, it would be, but building an RPM with a PAR::Packer built executable doesn’t actually work, at least not on Red Hat Enterprise Linux 5 (RHEL5) or RHEL6.

Let me explain….

Take this script, hello.pl:

#!/usr/bin/env perl

use v5.22; # Use perl 5.22's features
use strict;
use warnings;

use autodie; # This causes issues on Fedora out of the box

MAIN: {
    say reverse (split //, "!dlroW olleH");
}

If I run it on most distributions, line 3 will give me an error (the OS Perl isn’t new enough). On Red Hat/Fedora installs, if I changed line 3 to support older perl interpreters, the inclusion of autodie in line 7 will error out, unless that module has already been installed.

For instance:

perldemo:demo$ ./hello.pl
Perl v5.22.0 required--this is only v5.18.2, stopped at hello.pl line 3.
BEGIN failed--compilation aborted at hello.pl line 3.

Of course it works fine when I use my local perl interpreter (a perlbrew-built 5.22.1):

perldemo:demo$ perl hello.pl
Hello World!

So, I can create a packed executable, which runs great (at least on other systems with the same system libraries – the same limitations a an executable you dynamically link and build using C):

perldemo:demo$ pp -o hello hello.pl
perldemo:demo$ ./hello
Hello World!

Perfect! Note that I ran the executable hello, not hello.pl. The hello file is a self-contained executable that doesn’t depend on the system perl.

So what happens when I put this in an RPM package?  I won’t go through the steps of building an RPM, but will let you see what error you will get when you try running the executable:

perldemo:demo$ ./hello
Usage: ./hello [ -Alib.par ] [ -Idir ] [ -Mmodule ] [ src.par ] [ program.pl ]
./hello [ -B|-b ] [-Ooutfile] src.par

What the heck? It’s expecting a PAR file to be passed to it – the PAR, or Perl ARchive, of course contains the perl interpreter, your code, and your modules – for some reason the file can’t find it.

You can duplicate this without RPM by using strip on the output of pp on Linux:

perldemo:demo$ pp -o hello hello.pl
perldemo:demo$ ./hello
Hello World!
perldemo:demo$ strip ./hello
perldemo:demo$ ./hello
Usage: ./hello [ -Alib.par ] [ -Idir ] [ -Mmodule ] [ src.par ] [ program.pl ]
./hello [ -B|-b ] [-Ooutfile] src.par

What is going on? The strip removes debugging information and the like, but generally isn’t expected to change how your program runs – but in this case it seems to be removing a lot more. Running this same strip on OSX (Apple) is enlightening:

perldemo:demo$ strip hello
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: the __LINKEDIT segment does not cover the end of the file (can't be processed) in: /Users/jmaslak/pp/hello
perldemo:demo$ ./hello
Hello World!

So it appears Macs won’t strip the archive out of the extraction executable produced by pp – but Linux has no problem happily stripping it.

It turns out that standard configurations of rpmbuild on at least RHEL5 and RHEL6 automatically strip all binaries in the binary package (that’s the “standard” package you would install).

So how do you disable this evil? It turns out it’s not easy. On RHEL6, you add this near the top of your SPEC file (the SPEC file rpmbuild uses to create the RPM):

%global __os_install_post %{nil}

Now, rpmbuild won’t strip the file. But this doesn’t help RHEL5. So I added another line, that did what I want (assuming I don’t need a debug package, which I don’t in this case):

%define debug_package %{nil}
%global __os_install_post %{nil}

Sure enough, now my code does exactly what I want, even when I install the PAR executable via RPM:

perldemo:demo$ ./hello
Hello World!

Whew!

A Fix for Perl SSL on MacOS X 10.11

UPDATE (August 12, 2017) – Check out Tom Vander Aa’s comment in the comments. His solution is better than my solution in the original post – it has much less chance of breaking other stuff that uses OpenSSL.

ORIGINAL:

When trying to install some Perl Programming Language modules on MacOS X, using a perlbrew-built perl, I was getting some weird linking errors with various OpenSSL headers.  Here’s an example from Net::SSLeay:

laptop$ cpan install Net::SSLeay
...
Configuring M/MI/MIKEM/Net-SSLeay-1.72.tar.gz with Makefile.PL
CPAN::Reporter not installed.  No reports will be sent.
*** Found OpenSSL-0.9.8z installed in /usr
*** Be sure to use the same compiler and options to compile your OpenSSL, perl,
    and Net::SSLeay. Mixing and matching compilers is not supported.
...
cc -c   -fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -O3   -DVERSION=\"1.72\" -DXS_VERSION=\"1.72\"  "-I/Users/jmaslak/perl5/perlbrew/perls/perl-5.22.1/lib/5.22.1/darwin-2level/CORE"   SSLeay.c
SSLeay.xs:163:10: fatal error: 'openssl/err.h' file not found
#include <openssl/err.h>
         ^
1 error generated.
make: *** [SSLeay.o] Error 1
  MIKEM/Net-SSLeay-1.72.tar.gz
  /usr/bin/make -- NOT OK

I highlighted the interesting parts – I run homebrew on my Mac to manage development tools and the like. Sure enough, it had installed openssl already:

laptop$ brew install openssl
Warning: openssl-1.0.2e already installed

Hmmm, version 1.0.2e, which doesn’t look like the 0.9.8z that Net::SSLeay found.

A bit of Googling and scratching my head and I found the magic incantation:

laptop$ brew link openssl --force

Once that was done, I could successfully install Net::SSLeay and other SSL Perl modules.  I’m guessing that the links broke sometime during an OS upgrade. Hopefully this post will save you a bit of time!