Sunday, 9 April 2017

Ask HN: What are the pros and cons of PostgreSQL and MySQL?

For the past two years PostgreSQL has been my RDBMS of choice and has served me well. In fact I love PostgreSQL so much I decided to use it over the MySQL in my dissertation project, but for different reasons than the one exposed. Now someone has asked a sensible question on HackerNews and I am glad they did because the conversation unfolds nicely which tidbits from both sides.

I always got the impression (since I started using it in Prod in the late 90s) that the MySQL team had never worked on an RDBMS before, even as users, and didn’t really understand it. Back then they would say, you don’t need foreign keys, you can just enforce consistency in your application, you don’t need transactions, you can just handle it in your application, and so on and so on. Monty Widenius was very arrogant and thought he knew everything. Eventually they matured a bit and realized that actually, yes, the entire rest of the database community weren’t idiots, maybe there is something to these features, but now they needed to find a way to retro-fit them onto what they had and now – 20 years later – they still haven’t figured out how to smush it into their architecture.

Whereas the Postgres crew were fresh out of the Ingres project led by the genius Micheal Stonebraker, they had a very clear vision and a culture of doing the right thing, not the easy thing. There’s a steeper learning curve but once you have learned a few things you can easily guess the rest because it’s so consistent. For MySQL you really need a “guru” because there are so many dark corners and weird edge cases that make no sense, you just have to “know” them.gaius

Ask HN: What are pros and cons of PostgreSQL and MySQL?→

 

And that is also where I found this article that goes in depth on why MySQL might not be such a good idea. Here is a bad argument on why you should use MySQL over something else.

  • It’s getting better, so we might as well stay on it. It’s true, if you go by feature checklists and the manual, MySQL is improving “rapidly.” 5.6 is due out soon and superficially looks to contain a number of good changes. I have two problems with this line of reasoning:
    1. Why wait? Other databases are good now, not eventually.
    2. MySQL has a history of providing the bare minimum to satisfy a feature checkbox without actually making the feature work well, work consistently, or work in combination with other features.

Do Not Pass This Way Again – Choose something else →

Ah and this also reminds me about the list of features coming in PostgreSQL 10. If you haven’t read it yet, give it a chance.

 

Friday, 17 March 2017

Repository Showcase: Roadmap and resources on how to become a full stack dev, how to build a ‘safe’ A.I.

Starting this week I’ve decided to start a weekly ICYMI repository showcase blog starring a few of the repositories I’ve found this week and considered interesting.

This week it’s mainly about becoming a better me and nailing that interview with an interesting entry on how to build a safe A.I.

100+ Free resources for learning Full Stack Web Development →

The list below isn’t meant to be exclusive, it’s more so a collection of links that have helped me out along the way (and can hopefully help you). As you’ll see, I’ve focused on Javascript, React, and Node.js. There is also a wealth of information on interview prep and applying to jobs.

Web Developer Roadmap in 2017 →

I like this one because it helps you look at what paths you might want to take and paints a clearer picture of the available options. Features both frontend and backend, soon to have DevOps as well. Watch this repository for updates!

Below you find a set of charts demonstrating the paths that you can take and the technologies that you would want to adopt in order to become a frontend, backend or a devops. I made these charts for an old professor of mine who wanted something to share with his college students to give them a perspective.

Personal Guide to Software Engineering interviews →

Brush up your knowledge on different notions and principles. Data structures and algorithms explained, handful of useful resources.

Building Safe A.I. →

TLDR: In this blogpost, we’re going to train a neural network that is fully encrypted during training (trained on unencrypted data). The result will be a neural network with two beneficial properties. First, the neural network’s intelligence is protected from those who might want to steal it, allowing valuable AIs to be trained in insecure environments without risking theft of their intelligence. Secondly, the network can only make encrypted predictions (which presumably have no impact on the outside world because the outside world cannot understand the predictions without a secret key). This creates a valuable power imbalance between a user and a superintelligence. If the AI is homomorphically encrypted, then from it’s perspective, the entire outside world is also homomorphically encrypted. A human controls the secret key and has the option to either unlock the AI itself (releasing it on the world) or just individual predictions the AI makes (seems safer).

Not exactly a repository, but a Github Page, although @iamtrask makes the code available on his GitHub.

Thursday, 16 March 2017

Principles for C programming

Common sense advice that can be applied to any language. Definitely recommend the read and good C programming perspective.

Avoid magic. Do not use macros. Do not use a typedef to hide a pointer or avoid writing “struct”. Avoid writing complex abstractions. Keep your build system simple and transparent. Don’t use stupid hacky crap just because it’s a cool way of solving the problem. The underlying behavior of your code should be apparent even without context.

Had this happen so many times where code I was reading was using an obscure solution and spent hours trying to understad what was going on.

It’s more important that a novice could understand your code than it is to use some interesting way of solving the problem. Ideally, a novice will understand your code and learn something from it. Write code as if the person maintaining it was you, circa last year.

What more is there to say?

Do strict testing and reviews. Identify the different possible code paths that your changes may take. Test each of them for the correct behavior. Give it incorrect input. Give it inputs that could “never happen”. Pay special attention to error-prone patterns. Look for places to simplify the code and make the processes clearer.

Principles for C programming →

And while you’re there, check out his other articles on C programming and the lessons to learn from it.

Tuesday, 14 March 2017

Building and deploying Mojolicious projects guide

I recently had to write some Mojolicious documentation for my final year project and for $work and I decided to use some of the knowledge I gained to put this guide together. It will hopefully help others who might be confused about how to get started using Mojolicious with Carton and Plenv. By no means I believe it is complete and I know some more experienced Perl developers will want to correct some or point out some mistakes. 

This structure assumes your project is named “ExampleApp”, therefore controllers and other files will use this notation. In real examples, the files may be called something different.

Table of Contents


Project structure


Mojolicious Application Structure Info
lib/ The lib folder is the main folder, contains most logic, controller and models.
ExampleApp/

  • Controller/
  • [Schema/]
    • Result/
    • [ResultSet/]
  • Schema.pm

ExampleApp.pm

Each Controller has .pm file, and potentially sub-controllers which go in folders under the same name. Schema folder holds Model information (DB Schemas, Results and ResultSets)
[local/] CARTON OPTIONAL/NOT PART OF MAIN PROJECT STRUCTURE

This folder may appear on production machines. It contains local installations of dependency modules for this project. Managed & installed through Carton.

[log/] Logging folder, usually has a development.log and production.log file.
[development.log]

[production.log]

public/ All external files go in the public folder (Bootstrap, JS files, fonts, images etc). Can have any sub-structure.
script/ Scripts that can be run using the controllers and their actions. By default, it has the main application start script.
example_appn Main script, it starts the application by loading files from the lib folder.
t/ Tests folder, all tests go in here (if there are any). They all end in the .t extension, but may be grouped in folders.
templates/ Page templates (for the View) which are structured according to the Controllers.
layouts/

example_app/

Layouts are (re)usable views/templates. Files can contain placeholder or perl tags to render elements that are passed to the view down from the controller.
[vendor/] CARTON OPTIONAL/NOT PART OF MAIN PROJECT STRUCTURE

This is a Carton generated folder and may appear in some projects. Contains the local dependencies for this project. Should be

[cpanfile] Dependencies manager file, specifies what perl module/packages this project relies on. Makes installing dependencies easier as the dependency installer can go through the list and their dependencies as well automatically.
[cpanfile.snapshot] CARTON OPTIONAL/NOT PART OF MAIN PROJECT STRUCTURE

File auto-generated and used by Carton.

example_app.conf Main config file, the application reads from this file before it starts, it can hold details or params to be used in the Controllers. Sensitive data should not be added to this conf file.
[example_app.development.conf]

[example_app.production.conf]

[example_app.sample.conf]

Alternatively, because no sensitive data should be committed in the repository, one can include a sample file with the expected config file structure.
For each running mode, Mojolicious can read from the main config file as well as production/development if database connections details are different, for instance.

Square brackets [ ] denote optionality, meaning the folder or files may or may not be present. E.g. One can choose to not enable logging for the application, in which case they don’t need to create a log folder.

What is CPAN/MetaCPAN?

The Comprehensive Perl Archive Network (CPAN) is a repository of over 250,000 software modules and accompanying documentation for 39,000 distributions, written in the Perl programming language by over 12,000 contributors. CPAN can denote either the archive network itself, or the Perl program that acts as an interface to the network and as an automated software installer (somewhat like a package manager). Most software on CPAN is free and open source software.

Links:

CPAN can be installed as a Linux module under system Perl environment with

Or, if a Perl environment manager like plenv or perlbrew is installed, with

Plenv has a shortcut for installing cpanm too:

After installing cpanm, to use it for installing a specific module system wide run:

How does Mojolicious work with CPAN?


Mojolicious applications may use and re-use Perl modules available on CPAN. Some of these are written especially for Mojolicious, but most modules are written to be framework and project agnostic, so they can be “plug-n-play”.

Any Perl module (.pm file) may use other Perl modules by including them at the top of the time or inline by writing this line use Perl::ModuleName. This will work just fine if the user already has that module installed on their machine, but on new machines, the module may not be installed and the application will fail to start. That is why most Perl applications keep track of their dependencies in a cpanfile.

Most Perl applications and projects keep track of their dependencies in a cpanfile file. These can be installed with cpanm and/or Carton.

Mojolicious projects also uses the cpanfile to keep track of their dependency, but it’s up to the user to keep the cpanfile up to date. A user may request the general package or a specific version of it, in order to ensure the project runs correctly and newer versions don’t break functionality.

Cpanfile example


To install all dependencies for a project one can run the following command while in the project directory:

This command will install all dependencies globally. However there are other ways to manage dependencies, using Carton.

By default, all modules are installed under system Perl (the Perl version installed on the machine you are using). If you plan on using Plenv or perlbrew, make sure the modules are installed under the Perl environment and version you need, otherwise your application may not recognise they are installed.

What is Plenv?


https://github.com/tokuhirom/plenv

Use plenv to install, manage and pick a Perl version for your application and guarantee that your development environment matches production. Put plenv to work with Carton for painless Perl upgrades and bulletproof deployments.

When and why would one use Plenv?


Plenv manages multiple Perl environments. For instance, let’s assume the production machine runs Perl 5.20.1 and that cannot be changed because some other applications running on that server rely on it. Your new application requires a newer version of Perl, namely 5.22.0. Plenv can install and manage any Perl version and allow your new application to run using the newer version with little hassle.

It also helps developers test features locally, if their local version of Perl is different than the one in production.

How to install and use Plenv


A full Plenv installation guide is available here:

https://github.com/tokuhirom/plenv#installation

After installing a new Perl version, one can switch between them locally with

Or globally:

After any change or module installation it’s recommended you run:

The version of Perl the application will use can be forced/overridden by specifying the PLENV_VERSION Environment Variable when the application is run. E.g.

 

What is Carton?


https://github.com/perl-carton/carton

Carton is a command line tool to track the Perl module dependencies for your Perl application. Dependencies are declared using cpanfile format, and the managed dependencies are tracked in a cpanfile.snapshot file, which is meant to be version controlled, and the snapshot file allows other developers of your application will have the exact same versions of the modules.

With Carton installed, a developer can download all dependencies locally, and Carton will put them in the local/ directory. Carton will look into the cpanfile file for the dependencies your project needs.

What does Carton exactly do?


It does a couple of things and the developer can choose how they use it, but mainly it manages the dependencies. Here is in a few lines how it can be used and what it can be used for:

  • Carton keeps track of the packages and dependencies’ versions required for your application to run correctly (in the cpanfile.snapshot file)
  • Carton can download and install those dependencies in a local directory, without interfering with other projects that might need other versions of the same dependencies.
  • Carton can create a bundle with those dependencies that you can push as part of your project to make sure no matter what happens to remote source, you always have the versions you tested your app with and know it works.

One can choose to do only 1 and 2, or all three.

Why would I use Carton?


Some might question what are the benefits of using Carton over CPANM.

There are many reasons, but here are some:

  1. If your dependencies suddenly becomes unavailable to download from the internet, you can use the bundle you have with the right versions you know will work (or have tested the application against).
  2. You ensure your project’s independency of other factors (network offline, packages include breaking changes etc).
  3. You make sure everyone in your development team works with the exact same dependencies and their versions, putting an end to “But it works on my machine” arguments.
  4. Sometimes your dependency’s dependencies may become unavailable in which case you might end up with a broken dependency. Carton makes sure you have everything you need.

Carton can be used instead of cpanm and in conjunction with Plenv. Instead of cpanm doesn’t mean you don’t need cpanminus installed, as Carton uses it to fetch the dependencies. It only means you will no longer use cpanm as a command to install and manage a project’s dependencies.

Understanding Mojolicious deployment modes


As mentioned before and as the Mojolicious project structure shows, Mojolicious apps (often referred as Mojo apps) can be run either in development mode or in production mode.

Development is more verbose and allows better debugging, exposes sensitive data and dumps cookies, params and sensitive database connection, which is why it is not recommended for use in production. To run a Mojo app in development mode, use morbo command, which runs in foreground.

In contrast, production mode will fail without exposing sensitive data, but it also makes hot deployment easier. To run a Mojo app in production mode, use the hypnotoad command to spin up a process that will be tracked and run in the background.

To stop it, pass the -s option. To hot deploy it, after making your changes re-run the command above and hypnotoad will make sure you have zero downtime.

Based on the mode you’re running the application, Mojolicious will read the main example_app.conf file and the mode specific configuration file and combine them. You don’t need to have separate config files, but they can prove useful.

How to deploy a Mojolicious project?


Deployment outline:

 

Set up the system with Plenv.


This is not a mandatory step, so consult with the project README file or make sure you’re aware of the ways the machine you’re deploying to is set up.

For how to set up the Plenv environment refer to “What is Plenv?” section.

Pull the repository using git.

Mojolicious – and Perl applications in general – use a cpanfile to determine what Perl dependencies need to be installed. It is the user’s responsibility to keep the cpanfile updated with all the packages they are including in their applications.

Obviously Perl will need to be installed on the server that you will run the application on. On top of that, the application’s dependencies will need to be installed as well.

If using Plenv, make sure you prepend all commands with the plenv version you need (or set it locally/globally) as well as plenv exec.

To install the dependencies one will need to have cpanminus installed. Cpanminus is a dependency installer for Linux based systems. For more information please refer to “What is CPANM/MetaCPAN?” section.

Alternatively, if the developer used Carton to manage the dependencies, use the carton command to install the dependencies. Append with –cached if you want to use the included modules from the project bundle. For more information please refer to “What is Carton?” section

How do I know if the developer used Carton?


Check the project structure and look for hints of Carton (highlighted optional folders and file). Generally a cpanfile.snapshot is a clear evidence you should be using Carton to manage dependencies.

Always use Carton if available instead of cpanm. When using Carton prepend your run command with carton exec to let the application know it should use the Carton fetched and installed packages.

Configuration files


Mojolicious applications can run in two modes – development and production – and one can specify a main config file as well as separate deployment mode config files. Mojolicious will read both the main config file as well as the mode-specific config file and combine those. For more information please refer to “Understanding Mojolicious deployment modes” section.

Because they normally include sensitive data, the config files are normally not committed with the repository. Instead, a sample file is provided. Here is an example of a sample main config file which shows what goes.

The sample mode-specific should give you an idea of what the application’s config file needs to contain in order to run. This may contain database connection information, for instance.

Make sure your machine has access to the database you’re trying to connect to, otherwise you will get an error when loading the pages.

Run the web server


Mojolicious applications can be run as CGI or as web servers. Mojolicious comes with two solutions, morbo which is meant to be used in development and hypnotoad which is a production web server tool which allows really simple server reload and hot deployment, without ever taking it down.

All following commands can be run under different Plenv environments and in conjunction with Carton so you might need to append the right commands.

Development


For development go into the project top level folder and run

This will start the server in development mode, and by default running on port :3000. To stop it, just Ctrl+C it. You can override the port the application listens to by passing on the -l “https://*:3001” option to the command.

Production


For production run the following command:

This will start the server in production mode and keep in in the background. You make make changes to the files and folder, but they will only take effect when you reload/hot deploy the server again by re-running the same command. Hypnotoad will spin up another instance before it shuts down the previous one ensuring the server has zero downtime.

To stop the server just run:

If used with Plenv and Carton you’d probably need to run something like:

Where is the projects’ logs information?


By default, Mojo applications don’t save the log in production mode, whereas in development mode it just dumps it to the console. To make sure you save the log, create a log folder in the projects and the application will save all logs in either development.log or production.log file.

Please refer to the Project Structure section for more information on log folder.

For more information consult the Mojolicious Documentation.

Saturday, 25 February 2017

Check Uber estimated times and prices from command line

Lately I’ve found myself using Uber exclusively for traveling around when I need to. But I’ve also found myself waiting just a little bit longer to see if that surge price goes down. It just seems that there is always surge pricing, and it’s so annoying trying to refresh from the application because you never know if it’s a new price or the old one. But I’ve come to accept it because I trust it more than I trust regular cabs and it’s also a lot more convenient. So if I am in front of my laptop now I can use this, because I iz developer, right?

That being said, as a lazy person it pains me everytime open my phone, open the Uber app, type my destination, and see the estimated price, only for my inner, responsible, cost-cutting, fiduciary-self to end up taking the bus all the way home. Jae Bradley

You’re not the only one…

Check estimated time and prices for your next ride from the command line using Uber CLI →

Quite fun and also pretty good. Also check out these other nifty Uber projects to play around with that I discovered on ProductHunt:

However, ever since this news broke out, I am seriously considering ceasing the use of the app and joining the #DeleteUber movement. Just waiting on the independent review outcome.