Paul Graham is Wrong

In Paul Graham's latest blog post, he voices his support "to make immigration easier" to allow easier hiring of foreign tech workers by using a overly simplistic argument:

The US has less than 5% of the world's population. Which means if the qualities that make someone a great programmer are evenly distributed, 95% of great programmers are born outside the US.

US does rougly has 5% of the world population: 320 million at end of 2014 out of the world's 7.3 billion. But is comparing strict population appropriate? Are rural farming families in China likely to apply for technical jobs in the Us? No.

A better comparison maybe to compare number of college graduates. The data is less accurate, but this gives you a good picture:

The ratio of these numbers are much closer, far less dramatic.

Constrained for Talent, or Cheap Talent?

Graham argues that there are just not enough great programmers to go around and the immigrate programmers are being paid the same:

But if you talk to startups, you find practically every one over a certain size has gone through legal contortions to get programmers into the US, where they then paid them the same as they'd have paid an American.

Personally, I cannot imagine a start up that can affort this type of legal fees are start ups anymore. I would love to see the actual figures that back this argument up.

Immigration is Good

Ironically, I believe immigration reform is good, and it is good not just for the tech world. I like free markets. I like competitions. Within the startup world, I have hired and worked with great programmers all around the world.

But Local is Important

There is a lot to be said for having software developers that understand the local culture and market. To use a reverse example, can a US designer successfully design a UI for the Chinese market without a lot of local help? No.

What to do?

To solve the root of the program, America needs to pay more attention to STEM in school. We need to eliminate the gender gap in engineering. We need to teach more and better maths in elementary schools. Otherwise we are loosing the race to build a great talent pool.

Flask, REST and PiGlow on a RaspberryPi

The RaspberryPi (RPi) is the little computer that could. $40, runs a version of linux. But can you use it for "serious programming", deploy "real" applications? Of course !

As an excercie, I decided to put together a RESTful API server, and a web interface to control the fun PiGlow LED display board on a RPi. This project uses:

  • virtualenv
  • flask
  • flask-restful
  • angularjs (ok this is really off the RPi)
  • thread locking to control hardware access
  • and most importantly, it let me use the RPi+PiGlow board as a remote display any where with network access.

You can see the code and documentation here on github.

Arhitecture


This diagram shows the overall architecture:

Hardware - PiGlow Board

Get the PiGlow board from the PiHut if you are in the UK or Adafruit if you are in the US. This board provides 18 LED's with controllable brightness in 6 colors, arranged in a spiral pattern. Think of all the fun patterns you can generate. What makes the PiGlow board particularly great is that it is cheap ($15) and it plugs into the GPIO socket directly. (See Picture)

Basic Software

The PiGlow board is connected to the RPi via the i2c interface on the RPi. To control the pins in python, you can use the python-smbus python package at the lowest level, or you can use one of the many wrapper python classes.

i2c bus support

You also need to install i2c support first if you have not done so already. The original piglow example code has great instructions.

PyGlow.py library

There are many different python libraries written to support the PiGlow board. I like this version by Ben Lebherz here. 

Single Threading

The first step is to create a RESTful API server running on the RPi to talk to the PiGlow board. This will allow other software to control the LEDs on and off the RPi. One subtle but important note is that, since there is *one* actual PiGlow board, requests to update the board need to be done in sequence. We cannot have everyone writing to the i2c bus at the same time. So one important feature of the API server is that it single threads all the control operations, without blocking the API requests. i.e. the controlling of the board is done asynchronously from the actual API request/response.

The locking is acheive using a Lock object from the threading library.

Once a global lock object is created by calling threading.Lock(), all operations are wrapped with calls to acquire and release methods. In my code I use the python with statement to do this nicely.

# wait for lock becoming available if necessary
with lock:
    # here the lock is locked
    led_list[num - 1]['brightness'] = brightness
    pyglow.led(num, brightness=brightness)

# and the lock is unlocked

The asynchronous operations is done by putting the actual work in a separate thread from the thread that is handling the inbound REST request. The queue_command method in the PiGlowResourceMixin starts a thread to perform the operation, leaving the main thread to continue to response to the caller:
 

def queue_command(self, func, *args):
        """
        Queue function func with optional args in a separate thread.
        """
        h = threading.Thread(target=func, args=args)
        h.setDaemon(True)
        h.start()
        return h

REST API

Flask has a nice plugin call Flask_RESTful that makes writing REST API servers very easy. In our server, we subclass Resource from restful, which provides routing of calls and argument parsing. Each type of "things", LEDs, LED arms,  and LED rings (colors) are all addressable objects via the REST API.
For example: (LedAPI)

class LedAPI(PiGlowResourceMixin, Resource):
    """
        REST interface to control individual LED.
    """
    def get(self, led_id):
        """
        Get the brightness of a LED.

        (These are cached values, not necessary correct!)
        """
        return led_list[led_id]

    def put(self, led_id):
        """
        Set the brightness of a LED

        PUT /leds/:id

        URL Parameters:
        id=[integer] in the range of 1-18

        Optional:
        brightness=[integer 0-255]

        Data Parameters (optional):
        {
            "brightness": [integer 0-255]
        }
        """
        self.validate_led_id(led_id)

        parser = reqparse.RequestParser()
        parser.add_argument('brightness', type=int, default=0,
                            help='Brightness for this arm of LED')
        args = parser.parse_args()

        b = args.get('brightness')
        self.validate_brightness(b)

        self.queue_command(set_led, led_id, b)
        return led_list[led_id - 1]

 

Mixin

We also use a python feature called mixin to add common functionality to all the API interfaces. The validation methods are shared this way. And most importantly the queue_command method which provide the threading support is added to all the endpoints using this mixin.

The RESTful API Server

You can start the API server by just passing the file to python. For example:

python /usr/local/lib/python2.7/site-packages/piglowserver/piglowserver/pg_rest_server.py


API

The API server responses to a set of PUT requests to control individual LEDs, list of LEDs, "arms", and "colors". See the readme on github () for full documentation. But a quick example will give you an idea:

    # set arm 3 to brightness 50
    curl -X PUT -d brightness=50 http://localhost:5000/arms/3

    # switch on and off LED 7
    curl -X PUT -d brightness=100 http://localhost:5000/leds/7
    curl -X PUT -d brightness=0 http://localhost:5000/leds/7

    # switch on led 3 and 5 with brightness 10 and 200
    curl -X PUT -H 'Content-Type: application/json' \
        -d '[{"led_id":3, "brightness": 10}, {"led_id":5, "brightness":200 }]' \
        http://localhost:5000/leds

    # excute a starburst pattern
    curl -X PUT -d brightness=100 http://localhost:5000/patterns/starburst

    # turn everything off
    curl -X PUT http://localhost:5000/patterns/clear

 

CORS / CSRF Issues

The REST server is designed to be access either directly from another application, or from a web based application. Using our angular one page application (localhost:8000/app) as an example, that web page contains client side javascript code that access the REST server via the API.

However, all modern browser consider this type of access, where a web page served from one server try to access resources on another server, it performs a cross site request forgery ( CSRF ) check. Therefore the REST server is setup to add a CORS header "Access-Control-Allow-Origin: *" to all responses, allowing this type of cross site access.

The Web Application Server

The REST API is designed for used by another software applications. You can call it from another computer system, using the PiGlow board as a display station remotely. To demonstrate this connection, this project also include a very simple web application server, pg_control.py. You can run this server, also a flask application, and control the REST server remotely. The pg_control server can be run on the same RPi, a different RPi, or on a completely different machine.

python /usr/local/lib/python2.7/site-packages/piglowserver/piglowserver/pg_control.py

The Javascript Client

You can also run the simple angularJS powered single page javascript client to control the PiGlow board. Just point your browser to the /app URL in the pg_control server above.

A Command Line Weather Display Client

Finally, we have a simple example to query Yahoo weather for the temperature in a simple Python script, and display the result with LEDs. Look in the examples/weather.py script. You will need to modify the script to access your particular REST server's IP address, as well as put in your own city name.

Because I am running it on a mac, I have included a plist file that you can use to schedule the weather client to run every hour on your mac.

For the Impatiences

If you just want to install all the software and try it, follow these steps:

  1. Enable the i2c support, following these instructions
  2. Add your user ID ("pi" normally) to the i2c user group with sudo adduser pi i2c
  3. Install Ben's library with:  pip install git+https://github.com/benleb/PyGlow.git
  4. Install my software pip install git+https://github.com/pkshiu/piglowserver.git
  5. Install additional python packages required for the piglowserver using my requirement files: pip install -r /usr/local/lib/python2.7/site-packages/piglowserver/requirement.txt
  6. This will install three common python development tools: flask, flask-restful and requests onto your system.
  7. Run the API server: python /usr/local/lib/python2.7/site-packages/piglowserver/piglowserver/pg_rest_server.py (it will start listening on port 5000)
  8. Run the web application: python /usr/local/lib/python2.7/site-packages/piglowserver/piglowserver/pg_control.py (it will start listening on port 8000)
  9. Point your browser, on the RPi or on a different machine, to the web application:  <RPi_IP_Address>:8000
  10. Try it out!
  11. Also try the client side application: <RPi_IP_Address>:8000/app

Conclusion

This project demostrates the power of the RaspberryPi. While I also like the Arduino, the RPi has a rich development environment that allows for some powerful computing projects.

How to print multiple address labels from address book on the Mac

It is holiday cards time and every year I go through trying to remember to print address labels from my Mac OSX Contact app. The short answer? It is really simple.

Create a CSV File for Export from Contacts?

Contacts Export Options

No. This is not the way. But the old school me always start thinking down this road. Then I realized there is no "export to CSV" feature in Contacts. You can however drag contacts into Numbers and the contacts will automatically converts to a table.

Mail Merge directly from MS Word?

No. Or rather, you could do this. If you create a new document from the "label wizzard" template in MS Word on the Mac, it will ask for permissions to access your address book. Allow it. (You could also enable the access via the Security & Privacy settings in System Preferences.) But frankly I cannot figure out how to use the mail merge feature in MS Word. Too complicated.

The Mac Way

Print Labels from Contacts

Even as a long time Apple convert, I still am amazed by these obvious answers when using Apple product. You want to print multiple addresses from your Contacts (address book) on your Mac onto Avery 5160 label sheets? You just do the obvious:

  1. Open your contacts application
  2. Select the contacts you want to print: I have all my holiday cards contact assigned to a group, so I select the group and do a select all to highlight all the contacts
  3. Select File / Print / Labels and pick the right Avery label type
  4. print

No need to do export, conversions, merges. Just print labels.

Ban All Screenshots in Presentation

Do you put screenshots in your powerpoints or keynote presentations? Please stop now. Unless your presentation is actually about screen design, there is no real reason to include a screenshot of a software product or a website.

Why do people put screenshots in presentation? It is either to show progress, or features:

  • "Look at these screens ! We finished the product ( actually it barely works, that's why we cannot give a real demo)"
  • "Look at the menus ! We created so many useful features (but how often are these actually used?)"

This is all wrong

This point was driven home for me today when I went to an otherwise great presentation made by a town's IT department to a citizen group. The presentation content was great, but the slides were all  "bullet lists" and "screenshots". Because the audience consists of many older citizens, at one point someone raised his hands and say "I cannot read a single thing on your slide!"


Point made clear (sic)

This entire incident broke two fundemental rules about making a presentation:

  1. Know your audience
  2. Tell a story

Know your audience

In this particular example, we know that the citizen group consists of a large number of seniors. Many of them are also non technical. Yet the slides were all unreadable, full of small text bullet lists and screenshots. The materials were full of technology acronyms that the presenters did not explain, again until someone asked "what is GIS"?

Tell a Story

Instead of presenting a list of facts and show the webpages that the user can use to execute particular sets of functions, the presenter should have framed the presentation as a set of stories. Put the audience at the center of the story. How would the audience use this feature to achieve a desire goal?

An Example

As an example, part of the presentation is to highly a new GIS (Geographic Information System) enabled feature on a town website. A staff or a resident can now easily lookup information related to a particular property at once: perperty tax, voting district assignment, school district, neighborhood, distance to wetlands and recreational facilities, etc. The original data are stored in separate databases. Some of the geographic information, like wetlands and school districts, are geographic boundaries drawn on maps. So a special type of database and data processing is required to relate a property address to, say a district boundary. The technology is cool and useful. But showing a screenshot of the website with all the overlays is (1) unreadable, and (2) only exciting for perhaps a techie.

Instead, what if the presentation centers around how a resident may do to collect such information? How many "stops" does the resident need to make around the town offices to look up different information from different sources? But now all she has to do, or the town's staff has to do, is to visit the website, and obtain all the information she needed. 30 seconds instead of spending the morning at town hall.

The slides can be a set of pictures showing the rooms or counters at the town hall, or the piles of paper records and maps that she would have to look through, and the success case can be a slide of a happy user at her computer.

So please, stop putting screenshots in your presentation.

Know your Audience, and Tell a Story instead.