The Programmer’s Return: Taijutsu

When I first began preparing for the Flatiron School, I learned quickly to avoid max volume on my headphones when watching instructional videos. Programming requires that you spend a great deal of time in Terminal — which is a powerful interface built into your Mac that allow you to manipulate code, create files, yatta yatch.

Terminal reacts to set commands. As a basic example the command puts “Hello” returns “Hello”. The way Terminal works is that you type in the command and nothing happens, until you hit return (aka enter). Pressing enter evaluates whatever command you’ve setup and returns the appropriate response.

In more elaborate situations it is the feedback from the computer to let you know all the wonderful logic you just typed up does indeed equate to what you expected all along.

Additionally, consider this gif used on Learn.co to express a common misconception about programming:

Plainly, that as a programmer you spend 90% reading through code, following the logic. The remaining 10% of your time is actually spent manipulating it.

I think this is the unifying and underlying frustration that leads to The Programmer’s Return — the abnormal cubic force used to slam down the return key. If you watch any programming tutorial listen for the explosion that happens every time they press enter. It is the deafening result of programming mastery.

Many job interviewers will come equipped with tiny but dense rubber bands — requiring programmers to straddle their right pinky into the constraint in order to prove how effective they are at coding.

You’re Welcome!

A Beginner’s Beginner Guide to Scraping: Nokogiri Ninjutsu

In this post I will explain some of the fundamental concepts needed to utilize the Ruby gem Nokogiri. Gems in Ruby are comparable to plugins in WordPress. They are self-contained, little bundles of code you can drop into an application to gain additional functionality.

For more accurate reading please inject the phrase ‘To my understanding’ at the start of every sentence.

I followed Sam Callender’s wonderful tutorial, which you can find here. You can find the GitHub repo here.

Nokogiri allows you to scrape, or grab, information from a webpage or several webpages. Huh? Imagine you wanted to collect all your neighborhood restaurants to use in a web app you are building. Or if you wanted to grab all the content from a website and search through it to get a sense of what keywords they are targeting.

Nokogiri is often described as a tool to create an API when there is none.

In the tutorial referenced above we are collecting the first 100 Craigslist posts related to pets. We are taking each title and moving it into an excel document. You could also move it into your website’s database. The full potential of this gem is still over my head but I can tell it has potential to allow for very creative and fun implemenation.

So the url we are using for the tutorial is the same url you would see if you went to Craigslist and did a search for pets.

To understand the next step it is also important to realize you can right-click on any website and select view page source to reveal that site’s HTML. So as a simple example imagine this:

HTTParty Gem

The tutorial also utilizes another gem called HTTParty. This gem will allow us to grab the entirety of that source code (the HTML) and convert it into a string, aka some plain text we can manipulate.

So up until this point we have our dependencies (the gems we are requiring) and then a line of code that uses HTTParty to visit Craigslist and grab the page source code. So our ruby code to deploy a web scraping will look like this:

We set the variable page equal to the string we just collected. Using our example HTML to visualize what is happening: our page variable is pointing to something similar* to this:

*The pattern of back slashes is not accurate. The point being you receive all the original code bundled together into plain text.

Nokogiri Object

The next step is to convert the string we are storing in our page variable into a Nokogiri object. You will see below that our page variable is being called as an argument. The Nokogiri operation is being set equal to a new variable parse_page. So at this point our updated code is this:

If you run this process in terminal you will see that parse_page returns the original html, but as an object that we can operate on with Ruby. Which is cool and worth celebrating:

Parse

As of late I been using parse as my go to verb for taking any sort of action over a collection of code. The average person might use this realization and time to google the word parse, I prefer to share my dilemma.

Moving on, the next step is to parse our object — which it seems is just iterating over it and putting each desired bit of information into an array. So first let’s take a look at our updated code:

So we have an empty array that we will eventually store everything in on line 10. Then on lines 12 – 15 we have our iteration method.

The big thing to note here is that Nokogiri allows us to use the .css method on our new object which we are pointing to with the variable parse_page. So in the code the series of .css methods that take css selectors as arguments would be this:

“.css(‘.content’).css(‘.row’).css(‘.hdrlnk’)”

This is some crazy, but slightly informed guessing, but I think what is going on here is equivalent to saying this in css:

.content .row .hdrlnk

These CSS selectors are specific to Craigslist.com. As an example of what is happening, we will use our html from earlier:

If we wanted to grab the phrases “Blah blah blah” and “Yatta Yatch”, our CSS path could be:

#content p

Typically we use CSS to style a webpage, but using the language’s syntax we can select specific sections of HTML.

The CSS selectors used in the tutorial grab all the titles from Craigslist. The next bit of code converts what our links into plain text, using Ruby’s .text method. We see this on line 13.

Lastly we push each string of text, each headline in this case, into our newly created empty array pets_array.

In this particular tutorial the array is then sent into a .csv file — which can be opened in Excel. The updated and final version of the code would be this:

A csv would be a handy way to look at data or manipulate it.

Once the the code is in a Ruby array though we could do anything with it. Send it off to our database and build an application that tracks all things Craigslist pets.

Nokogiri Name

Nokogiri is a type of Japanese saw with teeth that face inward… you see… saw, scraping it all comes together.

nokogiri gem ruby

This post could have possibly been more informative if I was not more or less responsible for giving the world the Kyary gif(t) found above. You are welcome.

Resources & Thanks

Distilled
Engine Yard
Ruby B-word Book
My Brain

WordPress Guy Is Going To Code School, Extra Extra

Hi,

This is intended to be an open letter to friends, fambam, and clients. I will try to keep it short, while answering potential questions.

I am writing a blog post, crazy. I am also running my website on the default WordPress theme, boring.

Okay! I was accepted into the Flatiron School’s Web Development Immersive program. WooooOOOooOooo, I am pretty excited — I found out earlier today. The acceptance rate is low and we are within a week of when the program begins on November 1st, so this came as a bit of a surprise. Don don don…


 
I start saying cool a lot after this point to keep your interest and excitement levels high!

I will be learning Ruby, JavaScript, SQL, and more. Ruby is a programming language that is considered to be approachable and really pretty when you read it. I will be learning how to collect and manipulate information (data) and apply it under a variety of conditions. Cool! It is comparable to PHP which gives WordPress all the logic needed to operate.

JavaScript is anything on the web that visually makes you go whoa that is cool! JavaScript and WordPress or really any modern web platform work hand in hand.

SQL is a database, WordPress uses MySQL. It stores all your blog posts and images. Cool!

The program takes an approach to coding that mimics real life implementation and a strategy towards approaching programming that can be used to approach yet another programming language. Cool!

I will have to keep a blog while I am in school, I am guessing this will be it. So if you want to find out what I am up to, then come back here.

So friends, fambam, and clients this brings us to wow Saied I always knew that you were incredible but that is incredible.

WordPress Clients

Now that we are past that, we have arrived at ‘what about my favorite WordPress guy’. I have some solutions! Many of which are a plea for your patience.

The big take away here is I will not be replying to emails as quickly as I have in the past. If you have an emergency I will do everything I can to resolve it as quickly as possible.

The program will run until the end of April. Come May my schedule will loosen up. Until then if we have a maintenance package setup everything will operate as it has been.

I will most likely not be taking on new work until May. If you have a request please shoot it my way and if I can make it work on a weekend I will.

Alternatively, I have a friend who is 10 times better at this whole internet thing than I am. Well not like literally 10 times and I am taller than he is so… I will approach said gentleman this week and see if he is interested in helping with any miscellaneous tasks any of you may run into.

Saied, honestly I have no idea why you included me on this email… I don’t know, I’m celebrating.

Check this out: WP Help.Co, my website for monthly maintenance services.

That’s it! Have some questions. Worried?! Reach out to me!