Thursday, 14 September 2017

Configure ECS to use a docker image and host static files globally.

As a continuation of Hosting static HTML files locally with Docker, today I'm focusing on running that image in ECS.

This part wasn't complicated at all and really you can just follow ECS's tutorial to see your containerized application up and running in the cloud. Minimal hassle. I don't think that there's a point in rewriting the ECS tutorial, so I'll just do two things:

The next part should be quite exciting and it is to prepare an image to run a Haskell-powered page locally. Sounds like fun.

Stay tuned and happy hacking!
Piotr

Friday, 8 September 2017

Hosting static HTML files locally with Docker

In my last post I outlined my plan for Haskell, Docker & AWS development workflow. Here's the first part of my plan:

Prepare a basic Docker image to host static HTML files locally.

I'm going to cover two methods here:
  1. Use a vanilla HTML server image and map the resources to be hosted.
    This way you can make changes to the hosted HTML files and see them reflected in the browser without the need of restarting or creating a new container.
  2. Use a vanilla HTML server image to build your own image on top of it.
    This way you can generate dockerfiles which, when built, will include the hosted resources in the docker images they produce.
Let's take a closer look at the two methods.



Prearation

If you're completely new to do Docker, I highly recommend watching this video. 40 minutes well spent and in that time you will have the contents of this post explained in detail. I will be using Docker version 17.06.1-ce and Windows.

When you're ready, there's some preparation to be done for both methods: we're going to need a web server able to host static HTML files. One of the most popular choices is nginx and at the time of writing this, the latest stable version is 1.12.1. In this post I'm going to use the 1.12.1-alpine simply because the image itself is small.

Let's pull that image:

docker pull nginx:1.12.1-alpine


Method 1

Command:

docker run --name tiny-nginx -d -v c:\temp\:/usr/share/nginx/html:ro -p 80:80 nginx:1.12.1-alpine

Explanation:
Run a docker container named (--name) "tiny-nginx" as a daemon (-d), bind mount a volume (-v) containing the resource you'd like to host in the following way: "container-host-HTML-folder-path:container-HTML-folder-path" (see how my windows container host path - "c:\temp"*, neatly mixes with the linux container path - "/usr/share/nginx/html"?**), in a read-only manner (:ro), exposing container's port 80 to my container host port 80 (so that I can actually access the HTML page from my local machine) using image "nginx" tagged as "1.12.1-alpine".

*It is important to note that we are mounting folders and not individual files. **This folder was given in the nginx image documentation.

Windows is going to ask for your permission to share the selected folder:



And now you're set. Your shared and mounted folder is now being hosted by the tiny-nginx container and every time you make changes to the HTML file locally, they should be reflected in the browser every time you refresh the hosted page. This method is pretty neat and allows for rapid write & verify iterations.

Method 2

In this method we're also going to create a docker container based on an image, but this time the image we're going to use is going to be built by ourselves on top of an existing one: nginx:1.12.1-alpine.

To do that, we first need a dockerfile. This is going to serve as a definition of our image.

Code (file: dockerfile):

FROM nginx:1.12.1-alpine
MAINTAINER piotr - dot - justyna - at - gmail.com
COPY html /usr/share/nginx/html

Explanation:
Based on image nginx, tagged 1.12.1-alpine (FROM), create a new image with a given maintainer and while creating the image, copy the contents of the container host's "html" folder* to container's "/usr/share/nginx/html"** folder (COPY).

*It is important to note that we are mounting folders and not individual files and that the folder referenced as in the example, must be available in the current, working directory.
**This folder was given in the nginx image documentation.

This basic dockerfile can now be converted (built) into an actual new image.

Command:

docker build -t static-content-nginx:1.12.1-alpine .

Explanation:
Build and optionally tag (-t) and image (name:tag -> static-content-nginx:1.12.1-alpine) using a dockerfile found in the current folder (.).

Now the image should get added to the list of docker's images. We can now run it as if it was any other image pulled from the Internet (see the Preparation section).

Command:

docker run --name tiny-nginx -d -p 80:80 static-content-nginx:1.12.1-alpine

Note the similarities between this command and the one I used in Method 1. The only difference is that we're not mounting a shared folder. Simply because it has already been added to the image itself during the build phase.



Both methods should result in the file being hosted in the locally running containers. For now, there should be only one container hosting the HTML file at any given time. Results:


That's it for now and you can find the dockerfile together with some sample HTML to be hosted here. In the next part I'm going to cover how to use the image we created in Method 2 in ECS (and not Method 1 as sharing and mounting local resources would be quite tricky!) and publish my sophisticated HTML code globally.

Stay tuned and keep hacking! Piotr

Friday, 1 September 2017

Haskell, Stack, Docker, AWS and a notepad of your choice

I've been playing with Haskell on and off for the last few years and at this stage, while I still consider myself a newbie, I feel quite confident I could write a small application (*something*) and have it do relatively simple tasks for me. It's great to have your application published and running somewhere in the cloud, but the whole process of getting your code to the cloud is much less so. At least it has been in my case. What I like to do is more programming and less configuration and manual tasks!

Check now inactive fontbot for example. The way it is written is pretty basic and its iterations are as basic as one can wish for:

* write code
* build it with Stack
* copy the compiled code manually to a prepared EC2 machine

Basic.

But, as I'm here to hack fridges, I want more. I want the cycle to be more automatic and less manual. I want to be able to publish my built code anytime I'm happy with the build. Preferably to a Docker container hosted by AWS (not necessarily a single EC2 machine). My daily work does not require any interaction with Docker (so your feedback is much appreciated!), but since it looks like a very useful thing, I plan to explore it. I'm pretty new to Docker and the idea of ECS, so it should be good fun.

Let's sketch a plan of this exercise (now partially updated with the parts I've already finished):

At the end of this exercise I should have a setup where every time I push my code to a chosen branch, it gets deployed to ECS by the pipeline.

Stay tuned and keep hacking!
Piotr

Sunday, 27 September 2015

Hackage - releasing packages

You are ready to share your first Haskell package with the whole world. How to do it? It is not difficult at all and should take you no longer than 15 minutes. I took some notes whilte releasing my package: roller to help fellow Haskellers share their work with the community. Let’s dive right into it.

To begin with, run the following command in your package directory:

cabal sdist

This will try to prepare a hackage-acceptable package of your code. Depending on your package, first time you run it, cabal will complain a lot producing the something like this result:

Warning: Cannot run preprocessors. Run 'configure' command first.
Building source dist for roller-0.1.4...
Source tarball created: dist\roller-0.1.4.tar.gz

Cabal configure needed, let’s run it:

cabal configure

This will produce following results:

Warning: The package list for 'hackage.haskell.org' is 44.3 days old.
Run 'cabal update' to get the latest list of available packages.
Resolving dependencies...
Configuring roller-0.1.4...
cabal: At least the following dependencies are missing:
optparse-applicative >=0.11.0,
random >=1.0.1,
regex-applicative >=0.3

Cabal complains about dependencies being not up to date, let’s update cabal first:

cabal update

(this will take a while)

An now, let’s update those problematic packages beginning with:

cabal sandbox init

and followed by:

cabal install optparse-applicative
cabal install random
cabal install regex-applicative

Now, cabal configure should not complain anymore, let’s find out:

cabal configure

This should produce:

Resolving dependencies...
Configuring roller-0.1.4...

Looks well! We are now ready to prepare the package:

cabal sdist

Now cabal should produce this output:

Building source dist for roller-0.1.4...
Preprocessing library roller-0.1.4...
Preprocessing executable 'roller' for roller-0.1.4...
Source tarball created: dist\roller-0.1.4.tar.gz

At this stage your package is ready and can be uploaded to hackage, so please use this link when you're ready for that final step.

Good luck with your packages and happy hacking!

Saturday, 8 August 2015

Haskell - updating GHC and leaving the Haskell platform

This was my first GHC update and since it made me spend a couple of minutes scratching my head and deciding on how to approach this, let me share my experience in the form of a Q&A for the sake of brevity.

Q: When to update my GHC?
A: Whenever you think you need a newer version. A good example would be: new hackage packages stop working (give installation errors) and require new versions of core packages like base.

Q: I am using Haskell Platform, how do I update GHC?
A: Simply download the latest GHC and make sure your PATH points at the updated version (and not at Haskell Platform). Fear not. Stepping away from, otherwise very useful, Haskel Platform will give you much more freedom and control over the packages you're using.

Q: I have the latest version of GHC installed. How do I update my PATH?
A: If you've been using the Haskell Platform until now, your PATH should point at two things located in the Haskell Platform:
  • GHC
    C:\Program Files (x86)\Haskell Platform\2013.2.0.0\bin
  • mingw
    C:\Program Files (x86)\Haskell Platform\2013.2.0.0\mingw\bin
To leave the Haskell Platform's GHC and start using the new one, simply update the PATH accordingly (depending on where you installed the latest GHC):
  • GHC
    C:\Users\Piotr\Documents\Haskell\ghc-7.10.2\bin
  • mingw
    C:\Users\Piotr\Documents\Haskell\ghc-7.10.2\mingw\bin

This should get you access to the latest Haskell packages. Please share if your experience was different or if you follow a simpler process.

Happy hacking!

Friday, 31 July 2015

Haskell - playing with Hackage

OK, so you have just finished reading your first Haskell book, you understand most of (if not all!) compilation errors GHC throws at you and you start thinking creatively about the language. If you are at this stage and wondering what's next, welcome to my team! Not so long ago I finished Learn you a Haskell and, while reading The Haskell Road to Logic, Maths and Programming, I decided I was ready to start playing with Hackage. My package of choice was Rasterific - mostly because I wanted to get something on the screen quickly.

In this update I'm going to walk you through installing a package of your choice (+ all its dependencies) in your local cabal sandbox and using it in your own Haskell program. Let's get started!

First of all, let's make sure you have the latest versions of

  • Cabal
  • cabal-install
installed. To do that, simply run the following command from your command line:
cabal install Cabal cabal-install
This will install the latest versions of both packages.

Once it's done, let's update cabal packages using the following command:

cabal update

Now, a special note for the Haskell Platform users: the cabal you are using from the command line might not (and it wasn't in my case) be the one being updated! In my case, the cabal my PATH variable was pointing at was located in the Haskell Platform folder

...\Haskell Platform\2013.2.0.0\lib\Cabal-1.16.0
but executing those commands made the cabal install its latest version in
C:\Users\Piotr\AppData\Roaming\cabal\bin (version 1.22.6.0)
I had to come up with a solution and I ended up changing my PATH variable to use the cabal installed in my AppData.

Once you have the PATH variable pointing at the updated cabal, navigate to the folder you want to place your experimental project in (still using the command line) and type:

cabal sandbox init
This will create a folder-local sandbox environment for packages that could have otherwise damaged your global cabal repository. In case anything goes wrong in the sandbox, you can always delete it and start fresh - no need to modify your global set of packages.

Once you have the local sandbox prepared, you are ready to install your package of choice and its dependencies. Just in case, make the first run dry (just list the required dependencies):

cabal install Rasterific --dry-run
You should see something like this:
Resolving dependencies...
In order, the following would be installed:
base-orphans-0.4.0 (new package)
bytestring-0.10.6.0 (new version)
Win32-2.3.0.2 (new version)
binary-0.7.5.0 (new version)
text-1.2.1.1 -integer-simple (new version)
hashable-1.2.3.3 (new version)
nats-1 (reinstall) changes: hashable-1.1.2.5 -> 1.2.3.3
time-1.5.0.1 (new version)
directory-1.2.3.0 (new version)
FontyFruity-0.5.1.1 (new package)
mwc-random-0.13.3.2 (new package)
unordered-containers-0.2.5.1 (new version)
semigroups-0.16.2.2 (reinstall) changes: bytestring-0.10.0.2 -> 0.10.6.0,
hashable-1.1.2.5 -> 1.2.3.3, text-0.11.3.1 -> 1.2.1.1,
unordered-containers-0.2.3.0 -> 0.2.5.1
bifunctors-5 (new version)
vector-algorithms-0.7 (new package)
void-0.7 (reinstall) changes: hashable-1.1.2.5 -> 1.2.3.3
contravariant-1.3.1.1 (new version)
comonad-4.2.7.2 (new version)
profunctors-5.1.1 (new version)
semigroupoids-5.0.0.2 (new version)
free-4.12.1 (new version)
zlib-0.6.1.1 (new version)
JuicyPixels-3.2.5.3 (new version)
Rasterific-0.6.1 (new package)
Warning: The following packages are likely to be broken by the reinstalls:
linear-1.18.0.1
force-layout-0.4.0.0
diagrams-contrib-1.3.0
diagrams-1.3
diagrams-lib-1.3
diagrams-svg-1.3
diagrams-core-1.3
active-0.2.0.1
lens-4.9.1
contravariant-1.3.1
semigroupoids-4.3
profunctors-4.4.1
free-4.11
kan-extensions-4.2.1
adjunctions-4.2
monoid-extras-0.4.0.0
dual-tree-0.2.0.6
bifunctors-4.2.1
comonad-4.2.5
bytes-0.15
Use --force-reinstalls if you want to install anyway.

Once you're happy with required dependencies, simply run:

cabal install Rasterific

If everything goes well, you are ready to start playing with the package! You can find Rasterific documentation here. I used it to write a small program drawing rectangles into a file:

import Codec.Picture(PixelRGBA8( .. ), writePng)
import Graphics.Rasterific
import Graphics.Rasterific.Texture

main :: IO ()
main = do
    let backgroundColour = PixelRGBA8 234 247 217 255 -- Cilantro Creme
        foregroundColour1 = PixelRGBA8 195 214 170 255 -- Mint Sherbert
        foregroundColour2 = PixelRGBA8 142 168 108 255 -- Pesto Paste
        foregroundColour3 = PixelRGBA8 77 100 45 255 -- Simple Green
        foregroundColour4 = PixelRGBA8 40 58 16 255 -- Dark Water
        image = renderDrawing 400 400 backgroundColour $ do
            withTexture (uniformTexture foregroundColour1) . fill $ rectangle (V2 50 50) 145 145
            withTexture (uniformTexture foregroundColour2) . fill $ rectangle (V2 205 50) 145 145
            withTexture (uniformTexture foregroundColour3) . fill $ rectangle (V2 50 205) 145 145
            withTexture (uniformTexture foregroundColour4) . fill $ rectangle (V2 205 205) 145 145
    writePng "test.png" image

-- palette taken from: http://www.colourlovers.com/palette/110443/Summer_Grass
You can also find the code here. To build it, execute the following command (and this is where a local sandbox comes very handy):
ghc -package-db=.cabal-sandbox\i386-windows-ghc-7.6.3-packages.conf.d main.hs
Once the program compiles, you can run it and start drawing. Here is my result:

And that's it! You have just installed your first package and used it in your own program. Please leave a comment if the installation process was different for you or if you experienced some other problems.

Let the adventure with Hackage begin!

Saturday, 25 July 2015

Haskell - the power of type declarations

Haskell - a very interesting functional language, which continuously helps me become a better programmer and a more efficient thinker. One small example (exercise 1.24) I found in the book I'm currently reading (The Haskell Road to Logic, Maths and Programming) made me think: "wow, this is just great". This "wow" moment is something I would like to share with you today. I am going to show you that it's the type declaration and not the actual body of the function that you should consider your best friend while reasoning about the function's job. Haskell, as a strongly and statically typed language lets you do that. If you think it's crazy, please read on!

Let's look at a small program:

add2 :: Integer -> Integer
add2 x = (2 + x)

main = print $ add2 1 -- prints "3"
If you're a Haskell virgin, I hope you can still follow what's happening (if not, take a look at some other examples from e.g. Wikipedia to get a feel of the language). The add2's type declaration (first line) states:

I transform Integers into Integers.

Pretty simple. However, you might come across a slightly different interpretation:

I take one parameter of type Integer and return an Integer.

Which also works in our simple example. You might now ask: "ok, so now how do we write type declarations for functions which take more than one parameter?".
add x y = x + y
I remember asking this question myself and experimenting with constructs like:
add :: Integer Integer -> Integer
add x y = x + y
or
add :: Integer, Integer -> Integer
add x y = x + y
but none of them worked. At the time I just accepted that the compiler is smart enough to determine that the last "->" denotes the returned type and every other "->" simply chains parameters together. Weeks have passed and I started reading about lambdas, currying, partial application. I never looked back at those failed experiments from the beginning of my journey. Until now.

Let's return to our first function - add2. Exercise 1.24 (or its variant, adapted to fit my add2 example) makes you remove the parameter "x" and see what happens. Haskell allows for (and eagerly supports!) partial application of functions, so in our example, the additon is just partially applied (we only have one out of two parameters: 2). Not a big thing. Let's remove the "x" and look at the code now:

add2 :: Integer -> Integer
add2 = (2 +)

main = print $ add2 1 -- prints "3"
This is when I started connecting the dots. I expected the code to work, but I never thought of this:
  1. If my function (partially applying addition) works without the parameter, it really is nothing more than just an alias for that partial application of addition. This was a very powerful thought as it was always natural to me to associate the assignment operation with some sort of an order for the computer to "run the code on the right to give value to the code on the left". I started asking more questions:
    • If we don't need the "x" now, have we ever really needed it?
    • Is this going to change the way I think about code (not only the functional code)?
    • If so, how will all my programs look from now on?
    I am in the process of looking for answers to these questions and I find it very fresh and exciting.
  2. Let's look at the type declaration again - it hasn't changed! This made me think that my perception of type declarations was not very accurate. I realised that it serves as the promise of what the function is expected to achieve. The body obviously does the heavy lifting and fleshes the type declaration out, but it's the type declaration that helps understand the big picture. In the end, it does not really matter if your function takes one parameter and returns its processesed value or takes zero parameters and returns (or acts as!) a function that does it. What matters is the transformation defined in the type declaration. Implementation is just a set of various operations that should follow the principles outlined in the type declaration.

Learning Haskell (and functional programming in general) is a great experience which I can highly recommend to every software engineer. I plan to continue reading and understanding Haskell as it's a great source of inspiration and ideas. I've been writing some sandbox Haskell code in my free time, which you can find here and contribute if you please.

I hope this makes sense to you and if not (or if so!), please feel free to leave a comment in the section below.