Gianluca Arbezzano - GianArb2024-03-10T18:48:25+00:00https://gianarb.itGianluca Arbezzanohttps://gianarb.itgianarb92@gmail.comLinkerd jumped on the bandwagonBuoyant is the company https://gianarb.it/img/gianarb.png2024-02-22T06:10:10+00:002024-02-22T06:10:10+00:00https://gianarb.it/blog/linkerd-jumped-on-the-bandwagon<p>I am not here to say if service mesh is useful or not I am sure it depends.
The <a href="https://www.techtarget.com/searchitoperations/news/366570820/Linkerd-service-mesh-production-users-will-soon-have-to-pay">press “Linkerd service mesh production users will soon have to
pay”</a>
reports that we have lost another opensource project or at least there is a new
drama in town.</p>
<p>Nothing bad but it comes at the perfect time with the meme we are all
looking at <a href="https://www.reddit.com/r/github/comments/1at9br4/i_am_new_to_github_and_i_have_lots_to_say/">those days</a>. The timing is so perfect that I had to check the
calendar, it is April 1st.</p>
<p><img src="/img/i-am-new-to-github-i-have-alot-to-say.png" alt="I am new to github and I have a lot to say" /></p>
<p>If you want your exe you will need to pay Buoyant for that!</p>
<p>I am being too shallow, only the stable releases will be on Buoyant! In
practice they do what everyone else does, they have used GitHub as a platform
to do category creation, community, and so on. Now is the time for the company
to monetize.</p>
<p>Nothing to be surprised this is the evolution of the opensource ecosystem, or
at least that’s what company founded by VCs relaying on GitHub for marketing
want us to believe. We have all worked for some of those in the last 10 years.</p>
<p>Since I do opensource and I value this ecosystem I am wondering how such
decision can be taken and communicated so poorly, the definition of opensource
per se should avoid those standpoints. How can a company on its own define and
change the release management for an opensource project? The company is not
even called as the opensource project!</p>
<p>Can a single company can say something like this? What are the contributors and
maintainers doing? At this point is Linkerd sustainable as opensource project
behond Buoyant? Probably no.</p>
<p>The Cloud Native Computing Foundation guarantees for Linkerd, thankfully they
gather and organize stats about their opensource project. <a href="https://linkerd.devstats.cncf.io/d/5/companies-table?orgId=1">So let’s have a
look</a>, in 2023
they counted 128.856 contributions to the project, 112.721 from the same
company Buoyant Inc. followed by 1.400 contributions made by independent
contributors, 484 from the CNCF and 313 from Microsoft Inc. I won’t calculate
the percentage becauase it looks and unpleasant definition of opensource in my
opinion.</p>
<p>Anyway, this can be an opportunity for you if you are one of those independent
contributors. I spent a couple of minutes reading the <a href="https://news.ycombinator.com/item?id=39459102">HackerNews
thread</a> coming from this press
release and here some numbers:</p>
<blockquote>
<p>Their new offering (BEL) would be around $14k/mo for our org (though they
say discounts are available), with 90 days notice. That’s a rather large
chunk of change I didn’t request in our 2024 budget, for a cost-category
that didn’t exist before.</p>
</blockquote>
<p>If you want to market yourself as an alternative to Buoyant helping company
staying on top of their Linkerd installation, you just have to ask for less
than 2k USD a cluster and with that you are helping Linkerd to stay opensource
avoiding vendor lock-in.</p>
<p>NOTE: this is not a legal advice, I am not sure if you can do it, nowadays
opensource licenses are madness.</p>
<p>Open your editor, smash some HTML and CSS it is now the time to use that
service mesh related domain you bought a few years back!</p>
How I discover new codebasesStrategies I use when I want to contribute to new codebaseshttps://gianarb.it/img/gianarb.png2023-10-24T06:10:10+00:002023-10-24T06:10:10+00:00https://gianarb.it/blog/how-to-contribute-to-new-codebases<p><img src="/img/you-gen-a-new-job-no-docs-read-the-code-meme.jpg" alt="" /></p>
<p>Today is my time to be honest with you. I think this meme describes a lot of
places and codebases that I had to deal with or that I contributed to.</p>
<p>I don’t want to tell you why because there are many reasons about how you can
end up with a blob of undocumented code and I can write an article on its own
but I want to share the strategy I use to figure it out.</p>
<p>Why am I the right person to do so? Because I change job frequently, not
because of undocumented code obviously and I like to contribute to opensource
software that I use, and sometime I end up contributing to small undocumented
libraries, or to overly documented massive projects.</p>
<ol>
<li>Take a look at the CI/CD system</li>
</ol>
<p>Many applications have a basic CI/CD system, to run tests, to check code
formatting and sometime to build the software itself. When I don’t event know
the language because you I am contributing to a codebase developed in a
language I am not familiar with the CI/CD teaches a lot about the toolchain
that I need to have in order to be effective. Moving forward I tend to replace
tools I don’t like with alternatives I am more familiar with but at the
beginning CI/CD, makefiles, npm packages or equivalent files are gold. Worst
case if they don’t have unit tests or they don’t build their code in CI/CD I
end up knowing about what they use to format their code, it does not look much
but usually it drives to the required tech stack.</p>
<ol>
<li>Dockerfile</li>
</ol>
<p>Dockerfile are useful to figure out dependency tree and system dependencies
that can teach you a lot about the codebase you are dealing with. It is also
useful to figure out if my teammate are familiar with containers, or more old
style cmake and <code>./configure</code> kind of people.</p>
<ol>
<li>The entrypoint, look for that!</li>
</ol>
<p><code>fn main</code>, <code>func main</code>, <code>index.php</code> look for the entrypoint! If I see more than
one entrypoint I am in a monorepo, if there is only one it is a single
application. If I can’t find one maybe is a library but libraries should have
an entrypoint as well, so look for <code>Option</code> or <code>Configure</code> classes or objects.
If you find one the class using it is often the library endpoint.</p>
<ol>
<li>Run the test suite</li>
</ol>
<p>I like to run code locally when I can because it makes things a bit more real,
It validates that I figured out the right toolchain and that I am starting from a
trusted checkpoint.</p>
<ol>
<li>I need an easy win</li>
</ol>
<p>Why are you looking at such codebase? Do not miss the why! If it is your first
day at work and you got assigned to an apparently easy bug to fix this is your
goal, so try to figure out the right path, you know how to build the software
now, you know how to run the testsuite so leverage that when running the entire
application is still an unknown or is not possible.</p>
<p>I am not saying that tests should be the end goal for you, because there are
codebase with zero or unless tests, but they can be helpful at the beginning as a
north start, if they are unreliable or absent point 5 is still valid but there
is no escape the entrypoint needs to be discovered and used to validate that
your change has the right impact.</p>
<p>When I feel brave, I figured out the part of the code I want to contribute and
the are no tests I write a small scripts that imports such path so I can run
the subset of the code in isolation, quickly and repeatedly without too much
noise. At some point it ca even be turned into a unit test.</p>
Kubernetes is finally just an utilityKubernetes is a tool, not a religion. It can tech you a lot about about scalability, resiliency but if your buisness is not driven by this specific tool you should not overlook it.https://gianarb.it/img/gianarb.png2023-08-02T10:08:27+00:002023-08-02T10:08:27+00:00https://gianarb.it/blog/kubernetes-is-finally-just-an-utility<p>Kubernetes and cloud-native is a topic I spent a good time of my professional
life contributing to. I wrote some code, operators, articles, talks, libraries,
and I have been a member of the awesome Release Team for a few iterations.
Kubernetes helped me a lot to grow as a developer and improved my ability to
collaborate with people all over the world.</p>
<p>I know how to operate it, how it is written, the components it is made of, and
so on, but I do not work for a company that makes money out of it anymore.</p>
<p>I worked for cloud providers making money building software that integrates
with Kubernetes. I worked for companies that made money and took investments
saying: “Our solution runs natively on Kubernetes”. They didn’t age well and I
am not surprised at all.</p>
<p>If you are like me and your company does not profit from Kubernetes do not look
at it as something important. It is good to learn about it, to operate it but
that’s it, like you do with Linux, systemd, git, or everything else, because
that’s what it is. I spoke with many people who told me that Kubernetes is
complex, I got it, so it can’t be that complicated, can’t be more complicated
than systemd. There are cloud providers or companies that you can pay to get a
fully functioning Kubernetes endpoint to interact with up and running. I read
articles related to how the EC2 service works, and how and why they build it,
but that’s it. I don’t feel bad about using tools or services without knowing
all the details about how they are made.</p>
<p>I don’t want to discourage you from using Kubernetes or contributing to it. I
advocate for the good practices that Kubernetes enforces and teaches but that’s
the best it does. Today after two years of not touching it I installed kind,
the kubectl and I wrote 352 lines of YAML that I successfully applied to a
Kubernetes cluster because I am working with a potential customer that runs on
Azure and we picked Kubernetes as the common language, I think this is its
superpower. A technology capable of improving collaboration, and breaking
barriers is a gift that we should protect. And it does not require me to know
about CNI, CRI, CTO, and kubelet (can you guess the wrong one?).</p>
<p>Last week we expanded our solution from AWS, where I built the infrastructure
out of autoscaling groups, Launch Templates, EC2, load balancers, and duct take
to GCP where I decided to use GKE Autopilot driven all by Terraform. Not Flux
or Helm, two technologies that I never used but Terraform because the “IoC”
solution we have is 100% based on that and I didn’t feel the need for something
else. GKE because it makes sense, it is quick and I don’t have the experience
on GPC as I have on AWS to operate at a “lower level” in a reasonable time.</p>
<p>The solution we have on AWS is too expensive because there are a million tiny
details when building using simple components like the one I mentioned that are
painful to figure out at least for me, or at least not interesting from a
business point of view. This is why I am probably moving to ECS pretty soon.
Not EKS, because EKS does not look at simple as GKE Autopilot.</p>
<p>The developer I want to be and the one I like to work with put effort into
finding the right solution based on their current context, building all the
surroundings that will play a difference in the game we all have to fight with:
evolution and time. It requires knowledge and skills exceeding a specific
technology or trend. There are similarities with woodworking where complex
cuts or repetitive tasks require jigs, and those need to get built with
accuracy because they can drive the success of the primary project.</p>
You should avoid Meetup.comPlease avoid Meetup.com. You can do better on your ownhttps://gianarb.it/img/gianarb.png2022-11-24T10:08:27+00:002022-11-24T10:08:27+00:00https://gianarb.it/blog/avoid-meetup-com<p>A few years ago I decided to spend some of my time organizing a Meetup about
cloud computing in Turin, Italy. I got support from the Cloud Native Computing
Foundation to get a paid account to Meetup.com, food and drinks. I organized
various events in Turin and Milan as well. We had a good time and I worked with
different companies to help them share what they are building or passionate
about.</p>
<p>I work remotely and it was for me the best way to connect with other people in
my area working on similar challenges. The event was in English because a video
maker was there recording and mounting videos to share on the CNCF blog, or
with the companies or communities the speaker was involved with.</p>
<p>COVID-19 changed our daily routine drastically as you know and I embraced
virtual events as many others did. We gained good popularity and we reached 800
people registered to our meetup group, but we have missed the locality part of
all of this . In the meantime I decided to take a break as an active organizer,
leaving my spot to another person who supported me a lot during the day to day
operations. I left my role as CNCF Ambassador and in the meantime the CNCF
decided to move all those communities out from Meetup.com to their internal
platform.</p>
<p>I don’t want to comment on their internal platform, the migration was left to
the organizer, at some point we were maintaining events on both platforms
having the end of 2022 as the deadline to close the Meetup.com group.</p>
<p>In the meantime I tried to export the people who trusted me as organizer to
import them elsewhere but the attendees belong to Meetup.com and there is not
much you can do about it, Meetup.com locks you in.</p>
<p>What about deleting a Meetup group? Well, apparently you can remove all the
organizers and leave it in a limbo until Meetup.com removes it, or until
somebody else claims to be the new organizer.</p>
<p>Really! You work a couple of years to build a group of people who trust you,
your way of dealing with their time and when you decide that it is time to move
on the best Meetup offers is to leave those people on their own. Obviously just
as it happens with DNS the last day before termination somebody else claimed to
be the new organizer and took over the meetup group to share their own event.
Luckily it was a person I collaborated with in the past and I was able to
become the organizer again. It took me 2 hours to do the right things. I had to
kick out of the meetup group all 800 members one by one leaving an empty group
that nobody has any reason to claim.</p>
<p>It is my responsibility as organizer to take care of what happens to the people
who trusted me, and I think leaving them on their own to the first person that
finds itself in the right place, at the right time is wrong and you should not
trust a platform forcing this behavior.</p>
<p>Avoid Meetup.com, you can do better! Setup a mailing list, build a static
website on GitHub.com, write a few lines of whatever language you want to learn
and expose an HTTP server.</p>
<p>I like to share what I do and to experiment. In the last 10 years I organized
meetups, I tried to self-publish a book, I wrote on my blog and many of you
trusted me, leaving their emails to me and sharing their own time. The only
reasonable thing I can do is to clean up after myself when I am done. A few
years ago when I realized that the book I was trying to write didn’t make much
sense I deleted the 2000 people who registered to receive updates about it
because it is the right thing to do. If you find yourself in a similar
situation do the right thing, cleanup after yourself.</p>
From Ubuntu to NixOS the story of a mastodon migrationTwitter is not at its best. Developers are looking for an alternative as many others. Mastodon with its decentralized and feel of ownership is raising in popularity. I started with a hands crafted self hosted Ubuntu server because I felt the pressure about joining as early as possible but the end goal was to use NixOS for that. This is the story of how I moved my Mastodon instance to NixOShttps://gianarb.it/img/1280px-NixOS_logo.png2022-11-24T10:08:27+00:002022-11-24T10:08:27+00:00https://gianarb.it/blog/from-ubuntu-to-nixos-history-of-a-mastodon-migration<p>Do you know that Elon Musk bought Twitter for a lot of money? As a consequence many people are trying to figure out what to do. Developers quickly turned to Mastodon.</p>
<p>I decided to self host my server, you can interact with me on Mastodon as <a href="https://m.gianarb.it/@gianarb">@gianarb@m.gianarb.it</a>.</p>
<p>I do not have strong opinions about a decentralized system, I think it is another way to build a distributed system, experimenting with it is an opportunity, nothing more right now. I never liked the idea to sell my identity for free on social media but having a presence online proved to be crucial for my career and I don’t want to miss that.</p>
<p>Mastodon pushes many people, myself included, to think about: “should I host my own server?”. In my opinion it is an important question because it forces us to make our hands dirty again. We all know how comfortable GitHub pages are. You can set up your own static website in a minute, for free but it lowered my enthusiasm for technology because it makes things too easy. If you answered “yes” and now you are hands down trying to run your own Mastodon I hope you are having fun and that you are learning something that raises your excitement for how computers work. “Hosting more of my own things” was on my bucket list and Mastodon pushed me down the stairs.</p>
<h2 id="at-the-beginning-it-was-all-about-ubuntu">At the beginning it was all about Ubuntu</h2>
<p>NixOS has been my way to go for everything since the last two years, but I am not good at it. I tried to run my own Mastodon for a few days but I was not getting anywhere, got stuck trying to figure out how to properly manage secrets, and the machine lifecycle, how to deploy, how to interact with the tootctl, everything was a big unknown. Mastodon itself was a big unknown too. So I decided to step back and run my own instance following a random blog post:<a href="https://www.linuxbabe.com/ubuntu/how-to-install-mastodon-on-ubuntu"> “How to Install Mastodon on Ubuntu 22.04/20.04 Serves”</a>. Not sure if it is the best one out there but it gave me a Mastodon to play with in 10 minutes. Don’t need to tell me about infrastructure such as code, immutability and so on, this environment teached me all of this crap works, Mastodon is a bit more familiar and my end goal is still to figure out how to run it with NixOS.</p>
<h2 id="build-a-migration-plan">Build a migration plan</h2>
<p><a href="https://page.romeov.me/posts/setting-up-mastodon-with-nixos/">“Setting up your own Mastodon instance with Hetzner and NixOS”</a> by romeov explained how to get Mastodon running on NixOS. A few lines of configuration and the NixOS Mastodon Module configures Postgres, Redis, Nginx with TLS, and Mastodon itself for me. It is not the only way to go, the module supports running dedicated pools of those services as well but for my single user and single server configuration it is more than enough. So I started planning how to migrate my own server following the official <a href="https://docs.joinmastodon.org/admin/migrating/">Mastodon documentation</a> and it ended up looking like this:</p>
<ol>
<li>Provision a very basic NixOS instance (called beetroot from now on)</li>
<li>Stop mastodon services (web, sidekiq, stream) in the Ubuntu box</li>
<li>Take a backup of Postgres with the suggested command:<code> pg_dump -Fc mastodon_production -f backup.dump</code></li>
<li>Create a tar.gz archive for the system directory in Mastodon</li>
<li>Move the archive and the sql backup to beetroot via tailscale file:<code> tailscale file cp</code> public-system.tar.gz beetroot:</li>
<li>Get the two files from beetroot via tailscale: <code>tailscale file get .</code></li>
<li>Untar the system directory</li>
<li>Stop the mastodon systemd services, drop the mastodon database from beetroot and replace it with the backup from the Ubuntu server</li>
<li>Restart the mastodon services via systemd and have fun</li>
</ol>
<h2 id="how-it-went">How it went</h2>
<p>The plan was solid! <a href="https://pony.social/@cult">CULTPONY</a> looked at it briefly as well, so we are good!</p>
<p>But you know, in reality there are many unknowns. There is only one way to figure them out, time to stop making plans, it is time to break them!</p>
<pre><code class="language-nix">services.mastodon = {
enable = true;
localDomain = "PUT-YOUR-DOMAIN-HERE e.g. computing.social";
configureNginx = true;
smtp.fromAddress = "";
};
</code></pre>
<p>First, when I initialized the NixOS Mastodon module it starts an Nginx server because Mastodon requires TLS, it uses Let’s Encrypt for that and this requires the DNS record to point to the NixOS instance otherwise Let’s Encrypt won’t be able to close the loop, but I can’t point the DNS to a not yet ready instance because who knows if I am gonna be able to make it today, tomorrow or never! I decided to tell the Mastodon module to skip Nginx configuration for now setting<code> services.mastodon.configureNginx=false;</code></p>
<p>Technically there is <a href="https://discourse.nixos.org/t/nixos-deploy-in-a-vm-how-to-test-https-website-acme-lets-encrypt/8876">another way</a> to do it but it did not work for me and I still don’t know why. Let me know if you figure it out because it will be way more comfortable to get a self signed certificate so we can test without having to change DNS.</p>
<p>In the process of making the tar archive for the system directory I saw it contained a directory called cache, huge like multiple GBs. Cache to me means ephemeral, easy to rebuild, safe to wipe. So I did it! To be fair, I knew I was doing something stupid. And I knew the dirty way to go requires at least to move the directory, to keep it around until realization of the silly fact that cache means something important that should not be lost! Too late! I lost it, my Mastodon instance was then empty of all the avatars and profiles images, not a great start. After some googling and some struggling, <a href="https://github.com/mastodon/mastodon/discussions/21305#discussioncomment-4218030">I was able to build it back</a> (if you have a more official answer for this issue let us know there). My 2 cents: move the cache folder around, way easier than figure out how to get it back. Oh get another 2 cents, remember to check file permissions when you do this (guess why I know).</p>
<p>Everything now was set and ready to receive traffic, so I pointed the DNS to the new server, I set my host file to get routed to it quickly, I changed services.mastodon.configureNginx to true and I waited.</p>
<p>Ok! This is how it went after a day of struggling obviously! Last time I used postgres was probably 6 years ago. pg_dump, pg_restore are easy but I had to figure out how to authenticate properly, Ubuntu was set up to run over 127.0.0.1, the NixOS Mastodon Module by default provisions Postgres with <a href="https://www.postgresql.org/docs/current/auth-trust.html">auth trust</a> trust and with a socket entrypoint. It means that authentication does not require a password and it is based on a UNIX user. For example the Linux user Postgres owns and has access to the database owned and managed by Postgres. The NixOS Mastodon module creates a mastodon user in Linux with access to the mastodon file (the system directory for example) and with access to its own mastodon Postgres database. Nothing that looks like rocket science but still, it took me some time to figure it all out.</p>
<p>How to manage password in NixOS is a question I don’t feel comfortable answering yet and it blocked me at the beginning when I was trying to setup my own instance because I wanted to manage tailscale auth key for example automatically, or when thinking about how to manage the connection between mastodon web and postgres. Currently my answer is to avoid passwords. It works for now, but I know it won’t be the right answer for the following articles in this series that will probably title: “Mastodon monitoring a success story” where I will share how to configure the monitoring and observability pipeline for my instance with Grafana Cloud, but this is a story for another time.</p>
<p>Point 7 of the migration plan was about untar the system directory, but I realized I didn’t know where to place it. <a href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-apps/mastodon.nix#L32">Looking at the NixOS module</a> there is a path for that:</p>
<pre><code class="language-terminal">PAPERCLIP_ROOT_PATH = "/var/lib/mastodon/public-system";
</code></pre>
<p>But what does it look like? And what is PAPERCLIP_ROOT_PATH? Is it really what I think it is? It was not clear to me and only <code>var/lib/mastodon</code> was there in the system because the public-system folder gets created when Mastodon is actually in use. So I had to take a step back and I created a vanilla e2e working Mastodon instance to figure it out. At the end it <strong>obviously</strong> look like it should be, but who knew that!</p>
<pre><code class="language-terminal">[nix-shell:/var/lib/mastodon]# tree -L 2
.
├── public-system
│ ├── accounts
│ ├── cache
│ ├── custom_emojis
│ └── media_attachments
└── secrets
</code></pre>
<h2 id="show-me-the-code">Show me the code</h2>
<p>Currently I published the NixOS configuration for beetroot as part of my <a href="https://github.com/gianarb/dotfiles/tree/main/nixos/machines/beetroot">dotfiles</a> along with the other NixOS configurations for my Thelio workstation and for the Asus Zenbook I use at home. It uses <a href="https://nixos.wiki/wiki/Flakes">flake</a> and <a href="https://github.com/serokell/deploy-rs">deploy-rs</a>. It targets a Linode shared CPU virtual machine and that’s why, as you can see in the hardware-configuration NixOS detected Qemu as hardware.</p>
<pre><code class="language-nix">deploy.nodes.beetroot = {
hostname = "139.162.167.171";
sshUser = "root";
profiles.system = {
user = "root";
path = deploy-rs.lib.x86_64-linux.activate.nixos
self.nixosConfigurations.production;
};
};
</code></pre>
<p>Do not ask me about my deploy preference when it comes to Nix, deploy-rs is just the one I figured out, I may switch to Nixops because it is a bit more standard, they work similarly from a configuration standpoint but in theory deploy-rs is designed with profiles in mind, to deploy single users, something that I don’t think I need. But it works well enough for now.</p>
<p>If you look inside the flake.nix file you see two different nixosConfigurations, production and vm, both importing the same <code>configuration.nix</code>. Production is deployed via deploy-rs and vm is used for testing purposes with: <code>nixos-build build-vm -flake .#vm</code></p>
<p>I didn’t find a good use of it just yet, I am currently blocked by the acme certificate and because I am lazy. I am not sure if it is needed for Linode Shared CPU since it is a VM as well and it detects Qemu as a hypervisor. Time will help me figure it out.</p>
<p>At the beginning I developed this configuration outside of my dotfiles. Mainly because I didn’t know what to expect from it. Now that Mastodon is up and running and this configuration is in use I feel more confident. Even if I have a lot I want to do I decided to move it in my dotfiles to have access to other NixOS components there. I need to add a secret to authenticate to Grafana Cloud, probably with <a href="https://github.com/ryantm/agenix">agenix</a> in its own private repo imported via flake so I won’t have my password shared with you all (forgive me, it is not you, it is me or something else I don’t know), I want to move the cache directory and postgres data to a ZFS pool as well, but not now, right now I want to enjoy my running instance.</p>
<h2 id="now-what">Now what?</h2>
<p>This is everything I have learned for now migrating from Ubuntu to NixOS. I want to be clear, even if the core of this article looks like a bunch of mistakes I am not frustrated, I think the NixOS Mastodon Module is comfortable to use and well written. The challenges I described come from a rusty and inexperienced ops person. The module lacks documentation around operational experience, how to use it and what it provides but it is reasonable and I hope those notes will help to improve it and will push me to contribute back to the official documentation.</p>
<p>When I mentioned Prometheus and Grafana I shared that I am thinking of writing a series of posts about this topic, those are the one I have currently ongoing:</p>
<ul>
<li>Monitoring success story (probably with a deep dive in Password management on its own)</li>
<li>NixOS configuration, GitOps and machine lifecycle (this is about how I manage my NixOS configuration, how I deploy NixOS and so on)</li>
<li>Data management with ZFS</li>
<li>Mastodon update from 3.x to 4.0</li>
</ul>
<p>Your support and interest will push me forward writing all of them so let me know what you think about this one, the following topics and if you would like to read something else, like my journey with Linode, since I decided to try it out running this Mastodon instance there.</p>
<p>I would like to thanks all the writers behind the documentations, articles, GitHub discussions I have linked, and all the GitHub issues, StackOverflow questions, and GitHub repositories I have looked at to resolve my unknowns, sharing is caring! Thanks <a href="https://hachyderm.io/@hazelweakly">@hazelweakly</a> for your early review!</p>
My workflow with NixOS. How do I work with itIn the last two years I pick up NixOS as I tool I want to use. The learning curve is steep but I think I have a workflow that I likehttps://gianarb.it/img/1280px-NixOS_logo.png2022-09-12T10:08:27+00:002022-09-12T10:08:27+00:00https://gianarb.it/blog/my-workflow-with-nixos<h2 id="some-context">Some context</h2>
<p>Coding is fun when you can figure out the right workflow. There is nothing fun
when it comes to writing software in a way that is not sustainable or that does
not sparks joy.</p>
<p>I started to use Nix and NixOS almost two years ago, in a previous job in
a totally different context.</p>
<p>Back then we had to quickly and often provision operating system, build
software and so on. Since I moved back to write Software and to write Rust I
have to admit that building my code, or shipping operating systems is not
something I have to do very often, but I decided to keep learning and fighting
against NixOS because it fits my mindset.</p>
<p>Recently I resumed a few NUCs I keep in a box because everybody
deserves a home lab, and a good home lab deserves some netbooting, so it was
time to play with NixOS for something that is not my workstation or my laptop.</p>
<h2 id="the-workflow">The workflow</h2>
<p>Nix is code, finally. It means that there are libraries, you can import them,
run tests, and execute such code. YAML, Json in my experience, at some point
are a limitation, or they create friction, you ended up with an easy to break
template engine.</p>
<p>I decided to invest some time to figure out how to use flake. And this is where
I am so far:</p>
<pre><code class="language-nix">{
description = "A generic and minimal netbooting OS for my homelab";
inputs =
{
nixpkgs.url = "github:NixOS/nixpkgs/nixos-22.05";
};
outputs = { self, nixpkgs, ... }:
let
system = "x86_64-linux";
in
{
nixosConfigurations = {
generic = nixpkgs.lib.nixosSystem {
inherit system;
modules = [
./configuration.nix
];
};
};
packages.${system}.netboot = nixpkgs.legacyPackages.${system}.symlinkJoin {
name = "netboot";
paths = with self.nixosConfigurations.generic.config.system.build; [
netbootRamdisk
kernel
netbootIpxeScript
];
preferLocalBuild = true;
};
};
}
</code></pre>
<p>I am not the right person to tell you what all of this does because I am not an
expert and it is the outcome of many videos on YouTube, questions on
discourse.nixos.org, articles and beers, a lot of beers.</p>
<p>The <code>output</code> part describes what I want to build and as you can see there are
two outcomes. One is a <code>nixosConfigurations</code>, potentially it can contain more
than one NixOS description but right now I have a single one called <code>generic</code>
and as you can see it imports a module called <code>configuration.nix</code>. You can see
it as a ready to go NixOS provisioned as I want. This is 99% a copy paste of a
traditional <code>configuration.nix</code> file as you may know them. The one I use comes
from <a href="https://nixos.wiki/wiki/Netboot">“Netbooting Wiki” in NixOS.org</a>.</p>
<pre><code class="language-nix">{ config, pkgs, lib, modulesPath, ... }: with lib; {
imports = [
(modulesPath + "/installer/netboot/netboot-base.nix")
];
users.users.root.openssh.authorizedKeys.keys = [
"ssh-sfdbsrbs"
];
## Some useful options for setting up a new system
services.getty.autologinUser = mkForce "root";
environment.systemPackages = [ pkgs.tailscale ];
networking.dhcpcd.enable = true;
services.openssh.enable = true;
services.tailscale.enable = true;
hardware.cpu.intel.updateMicrocode =
lib.mkDefault config.hardware.enableRedistributableFirmware;
systemd.services.tailscale-autoconnect = {
description = "Automatic connection to Tailscale";
# make sure tailscale is running before trying to connect to tailscale
after = [ "network-pre.target" "tailscale.service" ];
wants = [ "network-pre.target" "tailscale.service" ];
wantedBy = [ "multi-user.target" ];
# set this service as a oneshot job
serviceConfig.Type = "oneshot";
# have the job run this shell script
script = with pkgs; ''
# wait for tailscaled to settle
sleep 2
# check if we are already authenticated to tailscale
status="$(${tailscale}/bin/tailscale status -json | ${jq}/bin/jq -r .BackendState)"
if [ $status = "Running" ]; then # if so, then do nothing
exit 0
fi
# otherwise authenticate with tailscale
${tailscale}/bin/tailscale up -authkey tskey-really
'';
};
networking.firewall = {
checkReversePath = "loose";
enable = true;
trustedInterfaces = [ "tailscale0" ];
allowedUDPPorts = [ config.services.tailscale.port ];
};
system.stateVersion = "22.05";
}
</code></pre>
<p>The only difference compared with a traditional non-flake configuration is the import:</p>
<pre><code> imports = [
(modulesPath + "/installer/netboot/netboot-base.nix")
];
</code></pre>
<p>Flake provides the utility variable <code>modulesPath</code> as a shortcut for accessing
the nixpkgs modules described as flake input.</p>
<p>This OS does a few simple things:</p>
<ul>
<li>Setup a public ssh key for the root user that I can use to ssh into the server.</li>
<li>It register itself to Tailscale</li>
</ul>
<p>The output <code>nixosConfigurations</code> is used via <code>nixos-build</code>.
It took me some time to figure out that <code>nixos-build</code> used in the right wat does not replace my current operating system.
Do not run <code>nixos-build switch</code> if you won’t want to screw up your local NixOS OS! Instead you can build this
operating system in the <code>./result</code> directory via:</p>
<pre><code>$ nixos-rebuild build --flake .#generic
</code></pre>
<p>A single configuration can describe different NixOS, that’s why you have to
identify what you want to build with ` .#generic`.</p>
<p>The second output builds the same OS but it shapes the content of the
<code>./result</code> directory as I want it (I am not sure if I need it but this is what
the NixOS netbooting wiki does, so far so good).</p>
<p>To build it you can use <code>nix build</code>:</p>
<pre><code>$ nix build .#netboot
</code></pre>
<p>Pretty cool! I can tar.gz that and ship it where I want. Straightforward.</p>
<h2 id="how-to-run-this-vm">How to run this VM</h2>
<p>Do you know how boring and time consuming it is to test a new operating system?</p>
<p>If you want to do it on real hardware you have to set it up, and if you want to
use QEMU you have a few days in front of you to remember all the flags you need, how
to bridge the guest with the host and who knows what. I tried for a few days
and I failed, until I discovered:</p>
<pre><code>$ nixos-rebuild build-vm --flake .#generic
building the system configuration...
Done. The virtual machine can be started by running /nix/store/dk4i22xmacnxxdmgvjhlyain5spb11yn-nixos-vm/bin/run-nixos-vm
</code></pre>
<p>Pure gold! If you run the <code>run-nixos-vm</code> script a QEMU virtual machine will
appear ready for you to test your operating system. Kind of cool! I can even
see it showing up in the Tailscale admin console!</p>
<p>A zero friction experience that boost my ability to try what I am working on.</p>
<h2 id="integration-tests">Integration tests</h2>
<p>Nix provides a testing framework, but I started to use it recently. It spins up
one or more virtual machines and assert that they work as expected. I wrote
a test that looks for the tailscale network inteface:</p>
<pre><code class="language-nix">let
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/0f8f64b54ed07966b83db2f20c888d5e035012ef.tar.gz";
pkgs = import nixpkgs { };
in
pkgs.nixosTest
({
system = "x86_64-linux";
nodes.machine = import ./configuration.nix;
testScript = ''
start_all()
machine.succeed("sleep 5")
machine.succeed(
"ifconfig | grep tailscale0",
)
'';
})
</code></pre>
<p>This test uses the same <code>configuration.nix</code> I used to generate my netbooting
NixOS. It starts a node called <code>machine</code> and via python script it runs the bash
command <code>ifconfig | grep tailscale0</code>. I am sure I can do better than <code>sleep 5</code>
but as I said, I am far away from being good at this.</p>
<p>You can use this approach to run assertions on multiple nodes,
here an example from Nix.dev <a href="https://nix.dev/tutorials/integration-testing-using-virtual-machines">“Integration testing using virtual machines
(VMs)”</a>.</p>
<h2 id="steep-learning-curve">Steep learning curve</h2>
<p>Everyone agrees that Nix and NixOS are not easy technology to pick up. And I
can confirm, there are articles, blogs, dotfiles available everywhere but they
look all different and it is hard to figure out if they are new, old or how to
apply them to your use case.</p>
<p>Flake is an attempt from the community to standardize all of that, and much
more. We will see!</p>
<p>It is also true that motivation and context can flat the curve. My plan is to
write more about this topic since I am trying to spin up and automated a home
lab.</p>
<p>I have to figure out how to do secret management but as soon as I have
it sorted out I will share my homelab configuration as I share my laptops
configuration in my <a href="https://github.com/gianarb/dotfiles/tree/main/nixos">dotfiles</a>.</p>
<p>Stay tuned.</p>
Website redesign and goodbye BootstrapI managed to remove Bootstrap from my website! You should not read this posthttps://gianarb.it/img/gianarb.png2021-12-17T10:08:27+00:002021-12-17T10:08:27+00:00https://gianarb.it/blog/redesign-goodbye-bootstrap<p>I managed to remove Bootstrap from my website! For me, it was the example of vendor lock-in.</p>
<p>A few years ago the trend was to use cloud provider but not enough to feel locked to a particular provider. In practice compute service was the only service allowed to be used. Everything else was an attempt from the devil to keep you down. The cause was lack of trust. Compute didn’t matter that much, it was not making anything more complicated, it was somebody else virtual machine.</p>
<p>Now it is clear that services like object store, managed databases, queue systems, machine learning or serverless are the secret of success when it comes to cloud providers. Because those services are stable and available for you to use quickly with zero operational effort. It gets harder to move to another vendor but there is not a lot you can do to avoid that. Motivation avoids vendor lock-in.</p>
<p>For me was the same, I tried many time to remove Bootstrap from my website but I never cared enough about the outcome, because I am not a designed and… really I don’t care.</p>
<p>Friday 17th December something changed! I had time and I was up for something boring, so I made it! I replaced Bootstrap with a couple of CSS classes.</p>
<h2 id="goodbye-navbar">Goodbye navbar</h2>
<p>I decided to remove the navigation bar at the top of the website. First, I don’t know how to make one on my own. Second, the number of pages were limited to three. Not enough to justify a real menu. Now a post contains the content and nothing more, very clean and I think it helps stay focused on what matters.</p>
<p>I left a small link to get back to the list of the other posts I wrote and that’s it. This is not a magazine and I won’t make any money out from this website, no reasons to drive you to other articles. Also, the people who reads what I write are good enough with computers to figure out what they want on their own.</p>
<h2 id="your-browser-is-cooler-than-me">Your browser is cooler than me</h2>
<p>No extra fonts, or font size, I tried to limit the number of html tags winning in accessibility. I trust your browser ability to interpret HTML and the font you have installed should be enough to read what I have to write! Let’s get back to simple things.</p>
<h2 id="adv-or-not-adv">ADV or not ADV</h2>
<p>I removed Google Analytics months ago. Is it time to remove ads? I didn’t make much money from this website, probably not enough to justify a banner and that javascript. I owned enough to run this website for other 10 years and I think it is enough! I reached sustainability! And you don’t need to thank me! I am the fist one using Brave as a browser, blocking noisy banners. I think the majority of the people reading my posts do the same.</p>
<p>The downside is that for me joining the Carbon network was a goal I had a few years ago. Because there is number you have to reach in order to enter the program. Adv is the only source of vanity number of this project and I kind of like to use $ for this even if it is a bit more than “nothing a month”. I don’t know, if you have feedback let me know!</p>
<p>My feeling is that 2022 will be all about finding other source of income that are not coming from my spending hours writing code. Not sure if spending time writing articles counts just yet. Probably not what I want.</p>
<h2 id="whats-next">What’s next</h2>
<p>Not much, my readers are not many, and this year I didn’t have much to write about. A website with fewest components will enable me to experiment a little bit more. An item for my invisible todo list is to write a yet another static site generator. The word needs more Rust code… who knows.</p>
<p>Probably it won’t happen. I don’t have special needs, but if will happen, you will notice!</p>
<h2 id="credit">Credit</h2>
<p>I think I am reading too much what Drew DeVault writes! The minimalism of this initiative comes from his <a href="https://drewdevault.com/">webiste</a>. My friend <a href="https://fntlnz.wtf">Lorenzo</a>, when it comes to CSS is a person I admire. He is good with eBPF as well but not as good!</p>
<p>Have a great Christmas!! Relax and do what you like with the people you love!</p>
How I started with NixOSI played with NixOS for the last couple of months. This is a story about how I picked it up, or how I should have done it.https://gianarb.it/img/1280px-NixOS_logo.png2021-10-01T10:08:27+00:002021-10-01T10:08:27+00:00https://gianarb.it/blog/how-i-started-with-nixos<p>I frequently change operating-system and distribution moving between macOS and Linux because I didn’t marry any of them yet.</p>
<p>Just before having a MacBook again, I was an ArchLinux user, a happy one. I have to admit it was not that different compared with other distributions, at least as a user. Yes, fewest packages installed, a few services, please don’t freak out, as I wrote I enjoyed it.</p>
<p>I see a value when it comes to describing as code your desires, learning from other people sharing their code, importing or copy-pasting it in different places.</p>
<p>Developers do that all day. I am a representative, I hope what I write will match my desires. After so many years I am full of hope.</p>
<p>With this in mind, Arch, Debian, or Ubuntu do not make a difference. It is all about the package manager. NixOS and Nix looked to me as a step forward in this sense.</p>
<p>I decided to end my vacation with macOS earlier. I picked up my personal Asus Zenbook 3 from its box to install NixOS.</p>
<p>Coming from ArchLinux the NixOS installation process is similar, we are on our own:</p>
<ol>
<li>Format disks</li>
<li>Write partition table</li>
<li>Mount partitions</li>
<li>And so on</li>
</ol>
<p>The main difference comes when you run <code>nixos-generate-config</code>:</p>
<pre><code># nixos-generate-config --root /mnt
</code></pre>
<p>The command tries its best to detect kernel modules from your hardware, mount points, and so on. This phase is a great time to start your first fight of many with NixOS.
The generated file will be in <code>/mnt/etc/nixos/configuration.nix</code> and <code>/mnt/etc/nixos/hardware-configuration.nix</code>. Open the generated file to can validate if they have sense. Don’t worry. It is a Linux distribution. If something is missed, it will tell us.
The <code>hardware-configuration.nix</code> file as the name suggests identifies your hardware.</p>
<p>Not everything can be detected yet, I use <code>luks</code> to encrypt my disks; the generated <code>hardware-configuration</code> needs a bit of help to figure it out.</p>
<pre><code class="language-nix"> boot.initrd.luks.devices = {
root = {
device = "/dev/nvme0n1p2";
name = "root";
preLVM = true;
allowDiscards = true;
};
};
</code></pre>
<p>Nix as a programming language takes a bit of practice, but NixOS is different. Many people share their configuration in GitHub, a boost in productivity.
I keep a list of NixOS configurations or Nix-related repositories that I look at when I don’t know how to solve a particular issue. <a href="https://github.com/gianarb/dotfiles/tree/master/nixos#credits">I really think you should do the same</a> because nobody wants to spend a day fixing its laptop, even worst if it is the one you use at work.</p>
<h2 id="start-simple">Start simple</h2>
<p>My end goal was to checkout my NixOS configuration as part of my dotfiles in a git repository. Too much when you don’t even know how NixOS works.</p>
<p>I have put aside this goal for a few weeks, and my new goal was to get my laptop working in all its part. The complicated part, that I didn’t solve in total is audio, it works but the volume control is not as good as it should be. You can check the configuration I use in my dotfiles but the solution does not matter.</p>
<h2 id="check-it-out">Check it out</h2>
<p>When I was happy with my configuration it was time to finally move it in its final destination. I joined the “stable era” for my Nix configuration, everything was good enough and it was not changing costantly. Perfect time for some refactoring.</p>
<p>I decided to use my <a href="https://github.com/gianarb/dotfiles">dotfiles repository</a> with a <code>nixos</code> subdirectory. This is the one I had when I first moved the configuration from my local environment to Git:</p>
<pre><code>$ tree -L 1 ./nixos
./nixos
└── machines
└── AsusZenbook
├── configuration.nix
└── hardware-configuration.nix
</code></pre>
<p>Those <code>*.nix</code> files are a copy of the one I have in <code>/etc/nixos</code>.</p>
<p>Now I had to teach NixOS where the new configuration are, the are various way, I decided to delete everything inside <code>/etc/nixos/configuration.nix</code>, leaving only an <code>import</code> to the configuration I moved as part of my dotfiles.</p>
<p>NOTE: I clone my dotfiles at <code>/home/gianarb/.dotfiles</code>.</p>
<p>I didn’t need <code>/etc/nixos/hardware-configuration.nix</code> and this is the content for my <code>/etc/nixos/configuration.nix</code>:</p>
<pre><code class="language-nix">{ config, ... }:
{
imports = ["/home/gianarb/.dotfiles/nixos/machines/AsusZenbook/configuration.nix"];
}
</code></pre>
<h2 id="pick-a-second-use-case">Pick a second use case</h2>
<p>I got a new Thelio System76 workstation (thanks to EraDB) and it was the perfect opportunity to re-use my fresh NixOS configuration, and my new skill.</p>
<p>At this point I am still working from my Asus Zenbook, but it is time to get a new <code>./machines/thelio</code> directory without a <code>hardware-configuratio.nix</code>, but only with the <code>configuration.nix</code> in there. The idea is to start extracting what you want to reuse from your first machine in its own files that can be imported everywhere you want.</p>
<p>I started from my user, because it is a common desire to reuse the same user across different machines. That’s why I have in my dotfiles a <code>users</code> subdirectory.</p>
<pre><code>gianarb@huge ~/.dotfiles (master=) $ cat nixos/users/gianarb/default.nix
{ config, inputs, lib, pkgs, ... }:
with lib;
{
# Define a user account. Don't forget to set a password with ‘passwd’.
users.users.gianarb = {
isNormalUser = true;
uid = 1000;
createHome = true;
extraGroups = [
"root"
"wheel"
"networkmanager"
"video"
"dbus"
"audio"
"sound"
"pulse"
"input"
"lp"
"docker"
];
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEKy/Uk6P2qaDtZJByQ+7i31lqUAw9xMDZ5LFEamIe6l"
];
};
}
</code></pre>
<p>I have imported it in both machines as we did previously for the all configuration and I splitted other applications like: <code>i3</code>, my audio configuration, vscode and so on. You can find all of them inside the <code>applications</code> directory:</p>
<pre><code>$ tree -L 1 nixos/applications/
nixos/applications/
├── i3.nix
├── sound-pipewire.nix
├── sound-pulse.nix
├── steam.nix
├── sway.nix
├── tailscale.nix
└── vscode.nix
</code></pre>
<p>Double checking that the refactoring is just a matter of re-building NixOS:</p>
<pre><code># nixos-rebuild test
# nixos-rebuild switch
</code></pre>
<h2 id="time-to-install-nixos-the-second-target">Time to install NixOS the second target</h2>
<p>I had everything I needed to re-install NixOS with my configuration into another target. It was time to setup a USB stick and boot Thelio from the USB.
The system I want is described as Nix configuration, the installation looks the same as the one we have done, or the one described in the documentation, but at this point, we do not need to generated configuration. We have our own one.
The only part we need, the first time, if we want is the <code>hardware-configuration.nix</code>.</p>
<ol>
<li>When you have booted from USB you can do what you have done previously, and what it is explained in the <a href="https://nixos.org/manual/nixos/stable/#sec-installation">NixOS installation guide</a>, format and parition disk.</li>
<li>When you have the disk layout done you can mount it to <code>/mnt</code> and you can clone/download somewhere the git repository with your nix configuration. I usually create an clone it where I want it to end up: <code>/home/gianarb/.dotfiles</code>.</li>
<li>Time to run <code>nixos-generate-config</code> as you do all the time</li>
<li>Replace <code>/ect/nixos/configuration.nix</code> with the <code>import</code>, copy the generated <code>hardware-configuration.nix</code> to your <code>machines</code> folder</li>
<li>Last step is to open the hardware-configuration and figure out if it has sense for your hardware.</li>
<li>When you are happy with it you can run <code>nixos-install</code> and it will install it from the configuration you have just declared.</li>
</ol>
<p>If it sounds like a convoluted process, it can be simplfied. But I didn’t yet invested into it yet! I don’t want to reinstall them all the time. You can read this article if you want to erase your laptop every day: <a href="https://grahamc.com/blog/erase-your-darlings">“Erase your darlings by Graham Christensen”</a>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>You just read about my journey with NixOS. With a centralized repository I can assemble, compile, and ship images to run on AWS, or ISO I can PXE boot.
I can build and compile a NixOS derivation that I can use as installation driver, for example cloning my dotfiles.</p>
<p>As next project I want to build an ISO that I can flash into a Raspberry PI who will act as media hub for my speakers playing Bluetooth audio or Spotify playlists via <a href="https://github.com/dtcooper/raspotify">raspotify</a>.</p>
How I tricked the cable mafia with PXE. Install OpenWRT on APU4dNo matter how many cables or dongle do you have they are never enough. The best you can do is to trick the system. I tried Pixiecore to PXE boot Alpine on my APU4d installing OpenWRT to it.https://gianarb.it/img/gianarb.png2021-04-06T10:08:27+00:002021-04-06T10:08:27+00:00https://gianarb.it/blog/home-made-router-msata-netboot<p>I am too lazy to buy a cable or another adapter. But not to buy an APU4d. A specialized for networking hardware with AMD Embedded G series GX-412TC, widely used for routers.</p>
<p>I got it directly from the manufacturer <a href="https://www.pcengines.ch/apu4d4.htm">PC Engine</a>. With a serial-USB cable and a <a href="https://it.aliexpress.com/item/32443776508.html">Huawei Lte miniPCI chip</a>.</p>
<p>I have also got the <a href="https://www.pcengines.ch/msata16g.htm">16GB mSata SSD module</a> because you never know, having a 16GB SSD, 4GB of RAM router sounds like an opportunity to run more tools on it!</p>
<p class="text-center"><img src="/img/apu4d.jpeg" alt="Picture of the APU4D board from PC Engine" class="img-fluid w-75" /></p>
<p>I assembled all of it nicely at my desk. It was too late when I realized I don’t know how to flash an mSATA SSD because I don’t have proper cabling…</p>
<p>No matter how big the box with all my cables and dongles is, I will never own the one I need. It is a mantra nobody can escape. The best you can do is to trick the system.</p>
<p>Luckily for me, the APU4d supports PXE booting, and we know how cool it is, perfect opportunity to try <a href="https://github.com/danderson/netboot/blob/master/pixiecore/README.api.md">pixiecore</a> and have some fun with netbooting.</p>
<p>It worked. If all of this sounds unreasonable, you need to remember that most likely you are right. But you know how much I like simple tools. Pixiecore was on my radar.</p>
<h2 id="get-what-you-need">Get what you need</h2>
<p>First of all, I installed Pixiecore. It is a Go binary, you can run it as a Docker container or, you can compile it with <code>go build</code> but I decided to use Nix shell:</p>
<pre><code class="language-bash">nix-shell -p pixiecore
</code></pre>
<p>In practice, it is a program that helps you to serve what a piece of hardware needs to PXE boot over the network, it servers IPXE and a TFTP server for example. It is light and not intrusive. You can keep your DHCP server, and if you like, even implement an API to drive how and what to PXE boot dynamically. Today I have to boot only one server in a very boring network. My solution is already too overly engineered. I decided to run it in static mode:</p>
<pre><code class="language-bash">sudo pixiecore boot ./vmlinuz-vanilla initramfs-vanilla \
--cmdline='console=ttyS0,115200n8 \
alpine_repo=http://dl-cdn.alpinelinux.org/alpine/v3.9/main/ \
modloop=http://dl-cdn.alpinelinux.org/alpine/v3.9/releases/x86/netboot-3.9.6/modloop-vanilla'
</code></pre>
<p>The first two arguments of the command line are the Alpine init ramdisk and the kernel. I got them directly from the <a href="http://dl-cdn.alpinelinux.org/alpine/v3.9/releases/x86/netboot-3.9.6">Alpine repository</a>.</p>
<p>The <code>--cmdline</code> option can be used to pass configuration to the operating system. The <a href="https://wiki.alpinelinux.org/wiki/PXE_boot">Alpine netboot wiki page</a> to know the various options supported by the init script.</p>
<p>Now that I have set the PXE distribution tool, I powered on the APU4d board. By default, it tries to boot from a couple of different devices. The last one is PXE mode.</p>
<pre><code class="language-console">sudo pixiecore boot \
./vmlinuz-vanilla initramfs-vanilla \
--cmdline='console=ttyS0,115200n8 \
ssh_key=https://github.com/gianarb.keys \
alpine_repo=http://dl-cdn.alpinelinux.org/alpine/v3.9/main/ \
modloop=http://dl-cdn.alpinelinux.org/alpine/v3.9/releases/x86/netboot-3.9.6/modloop-vanilla'
Password:
[DHCP] Offering to boot 00:0d:b9:5a:3e:10
[DHCP] Offering to boot 00:0d:b9:5a:3e:10
[TFTP] Sent "00:0d:b9:5a:3e:10/4" to 192.168.1.87:55360
[DHCP] Offering to boot 00:0d:b9:5a:3e:10
[HTTP] Sending ipxe boot script to 192.168.1.87:29233
[HTTP] Sent file "kernel" to 192.168.1.87:29233
[HTTP] Sent file "initrd-0" to 192.168.1.87:29233
</code></pre>
<p><code>192.168.1.87</code> is the IP the APU4 got from my DHCP. Everything is working and from the serial port I see Alpine booting, the <code>root</code> password is <code>root</code>! Classy!</p>
<h2 id="time-to-install-openwrt">Time to install OpenWRT</h2>
<p>I never used OpenWRT before. It is a Linux distribution for routers. You can even flash it to TP-LINK or Netgear devices if supported, at your own risk.</p>
<p>Anyway, since I am now running Alpine in memory on my APU4d I have a functional operating system and access to the device. I can use traditional tools like <code>dd</code> to write OpenWRT directly to disk, manipulate partitions and, so on… I followed the blog post <a href="https://teklager.se/en/knowledge-base/openwrt-installation-instructions/">“OpenWRT installation instructions for APU2/APU3/APU4 boards”</a> written by TekLager.</p>
<h2 id="conclusion">Conclusion</h2>
<p>My router looks up and running. I was able to reach the administrative Web UI. I didn’t use it yet because I have to relocate it to my new house. So I am sure you will read more about it in future articles.</p>
<p>Pixiecore was on my TODO list because those days hardware, datacenters automation are taking a good part of my daily working activity. Its support for external API makes it a great alternative to provide an installation environment like <a href="https://github.com/tinkerbell/hook">Hook</a> (the one we developed with <a href="httsp://github.com/tinkerbell">Tinkerbell</a>) without having to onboard the full Tinkerbell stack, in particular, I can avoid <a href="https://docs.tinkerbell.org/services/boots/">boots</a> when not needed.</p>
DIY Board management control for an Intel NUC: power controlThis is my first experimentation with reading and understanding a schematic. I hooked up an Intel NUC to a Raspberry PI to get control over its power lifecycle. I see it as a very simple board management control (BMC)https://gianarb.it/img/gianarb.png2021-03-14T10:08:27+00:002021-03-14T10:08:27+00:00https://gianarb.it/blog/homelab-diy-bmc-intel-nuc<p>I want to start this article with a disclaimer. What follows is not a tutorial or a guide, do what you want, but do not blame me if your fried your Intel NUC (they do not taste good).</p>
<p>When it comes to hardware and datacenters, I am not an expert. I was born and raised in the cloud, and recently, I joined Equinix and Metal (previously PacketHost). That’s why my interest changed, and I now have a disassembled NUC and a multimeter on my desk.</p>
<p>If you don’t know the origin of that PCB, I wrote a piece about my homelab for the Equinix Metal blog: <a href="https://metal.equinix.com/blog/building-an-ephemeral-homelab/">“Building an Ephemeral Homelab.”</a></p>
<p>Long story short, almost one year ago, straight after joining Equinix Metal, I got a couple of NUCs and Invidia Jetsons to play with, fully cabled in a 1U brick. Cool, but I have to admin only helpful for experimentation. It was cheap and the boards themself are old. But this is everything I need, something I can break without feeling too bad.</p>
<p>When it comes to fully flagged servers, you quickly learned that they are made of building blocks when an essential one is the board management console (BMC). Think about that as a small and low-consuming PC that manages the big brother, the actual server. When it comes to servers, you know they are loud, consuming a lot of power. That’s why you have a BMC that has the only responsibility to manage the expensive server. It can power control it, switching it on, off, and monitoring its status with metrics like volt consume, temperature, and so on. It can even select the boot device, for example. It is handy if you want to enter PXE mode, for example, to manage your server without touching it.</p>
<p>The BMC is wired to the server; you don’t have to power it separately; usually, you only have to hook it with an RJ45 to a switch. Extremely functional because those who have access to the BMC can take control of the actual server; it is an excellent idea to place the NIC in a dedicated VLAN.</p>
<h3 id="time-to-hack">Time to hack</h3>
<p>My homelab arrived cabled with a relay controllable from an outside board like an Arduino or a Raspberry Pi. Switching on or off the relay cuts the power brutally, almost like directly pulling the board’s power cable off.</p>
<p>NUC does not have a BMC and does not consume much, so there is no point in having another computer controlling them, but hey, this is my home lab, and we are after something here.</p>
<p>I downloaded my <a href="https://www.intel.com/content/dam/support/us/en/documents/boardsandkits/NUC5CPYB_NUC5PPYB_TechProdSpec11.pdf">board’s schematic</a> a few months ago, and from time to time, I look at it for inspiration. I studied electronics at high school, and Arduino got invented in my region, but I don’t think it counts.</p>
<p>I want to switch on and off my boards properly without leaving my desk because I am lazy. I want to use a Raspberry PI for this job because I can write code in any language I know. Spoiler alert for this prototype I have used ~5 lines of Bash.</p>
<p><img src="/img/bmc_pi_front_panel_spec.png" alt="Picture coming from the NUC schematic. It describe the pinout of the front
panel. It exposes a power switch and a few output pings to get power status from
the NUC" class="img-fluid d-block mx-auto" /></p>
<p>— Picture front panel schematics —</p>
<p>During one of many rounds of randomly reading the table of contents, I saw a Front Panel Header exposed from the NUC that says: “Power/Sleep LED Header”. It looks like there is a way to connect a LED to the NUC to see its status, fun! Nothing complicated: 1 the board is on, 0 the board is off. The LED can be replaced with a GPIO from the Raspberry PI (I used GPIO22) and hooked to a few BASH lines (as a prototype) to read the actual value from the NUC. I used this guide, <a href="https://raspberrypi-aa.github.io/session2/bash.html">“Bash Control of GPIO Ports.”</a> I used tmux so I can leave it running in the background:</p>
<pre><code class="language-sh">#!/bin/bash
tmux new-session -d -s power\_status
tmux send-keys "watch -n 1 'cat /sys/class/gpio/gpio22/value >> /tmp/current\_power\_status'" C-m
tmux detach -s power\_status
</code></pre>
<p>Now that the first mini-circuit is done and I am a bit more confident, I kept reading: “Power Switch Header.”</p>
<blockquote>
<p>Pins 6 and 8 can be connected to a front panel momentary-contact power switch. The switch must pull the SW_ON# pin to ground for at least 50 ms to signal the power supply to switch on or off.</p>
</blockquote>
<p>This sounds easy; I cabled PIN 6 from the NUC to GPIO 17 in the RPI and PIN 8 to ground, and with two bash scripts, I figured it all out!</p>
<p><img src="/img/bmc_pi_prototype.jpg" alt="A picture of a Raspberry PI cabled to a Intel NUC board to control the power
lifectycle" class="img-fluid d-block mx-auto" /></p>
<pre><code class="language-terminal">
$ root@raspberrypi:~/power\_swtich# ls
log.sh poweroff.sh poweron.sh
$ root@raspberrypi:~/power\_swtich# cat poweroff.sh
#!/bin/bash
echo "0" > /sys/class/gpio/gpio17/value
$ root@raspberrypi:~/power\_swtich# cat poweron.sh
#!/bin/bash
echo "1" > /sys/class/gpio/gpio17/value
sleep 0.2
echo "0" > /sys/class/gpio/gpio17/value
sleep 0.2
echo "1" > /sys/class/gpio/gpio17/value
</code></pre>
<h3 id="power-the-raspberry-pi">Power the Raspberry PI</h3>
<p>Half of myself likes the idea of getting a Raspberry PI or equivalent for each NUC, pretending that it is a BMC (I have other things I want to do with it, more at the end of the article). Either way, I like the idea to power the Raspberry PI from the NUC itself to save a power supplier and a cable. If you carefully looked at the front panel picture, you probably noticed that PIN 9 is a +5V_DC (2A), just enough to power an RPI via GPIO. But you need to know that GPIO unlikely the USB one does not implement any safety protection technique. If you supply an incorrect voltage, the RPI will burn.</p>
<p>Anyway, PIN 9 is not what I am looking for because it goes up to +5 V only when the NUC is on. We want to get the RPI powered on even when the NUC is off (but plugged with the power supply).</p>
<p>The NUC has a header called: “Auxiliary power connector” that does just what I need! I hooked it all up, and we have power.</p>
<p><img src="/img/rpi_bmc_auxiliary_power_spec_png" alt="This is an image I took from the NUC schematics. It describes how the
"Auxiliary power connector" works. And how it can be hook to another destination
of power to switch it on" class="img-fluid d-block mx-auto" /></p>
<h3 id="conclusion">Conclusion</h3>
<p>I can’t tell if this is or will never be a BMC, but I quite like where this is going, and I had fun. Short term, I can hook more NUCs to the same RPI and play with it. I can re-write the bash scripts you saw using some other languages exposing something like an HTTP API that I can interact with programmatically.</p>
<p>But I am after something better that I am not sure I can figure out. There is something else. I want to visualize output from the NUC. With Tinkerbell, I already have some control over the machine lifecycle because the NUC is capable of PXE booting. I can inject an in-memory operating system (Linux) and SSH into a NUC even if it does not have an operating system installed. But I want more; I want to look at the BIOS and things like that. An “easy” solution is the HDMI dongle. I can get an HDMI video capture, hook it up to the RPI and forward the NUC output with VNC or something like that; I can do something similar to forward what I type with a keyboard. A better solution is to use a serial console. Unfortunately, my board does not expose it. Joel, one of my colleagues at Equinix Metal, told me that my CPU most likely has it, and he is correct (accordingly with the CPU schematic), but the board does not have a header that I can use. But this is a story for a next article (if we will figure it out).</p>
<p><a href="https://pixabay.com/photos/measuring-equipment-electronic-2622334/" class="small">Hero image from Pixabay</a></p>
Nix for developersNix is slowly sneaking in my toolchain as a developer bringing back the joy of provisioning. Not as an exercise of translation between technologies and environment but as the art of building your own environment. https://gianarb.it/img/gianarb.png2021-01-25T10:08:27+00:002021-01-25T10:08:27+00:00https://gianarb.it/blog/nix-for-developers-sneaking-in-my-toolchain<h2 id="nix-is-slowly-sneaking-in-my-toolchain">Nix is slowly sneaking in my toolchain.</h2>
<p>Currently, a lot of my colleagues use Nix. It is a package manager that runs on Linux and macOS. It is versatile. I will show you more about it moving forward, but for now, think about it as a replacement for APT, YUM, and HomeBrew can work on both Mac and Linux. It is also a build system, but I didn’t use it much for it just yet.</p>
<h3 id="not-tight-to-an-operating-system">Not tight to an operating system</h3>
<p>For me, this is already a huge benefit. From time to time, for no reason, I end up switching from Mac to Linux and vice versa. It usually happens because I change the place where I work, and the policy forces me to make some weird decisions.</p>
<p>When I got off college, my parents bought me my first laptop; it was a Macbook Pro, but for the first two, three jobs, I used Linux because Mac was too expensive for my employers, and the main reason for owning a Mac was because back then I was not a fully flashed developers. I used to do video editing, playing with Photoshop, and so on. I quickly learned that I couldn’t match two colors nicely together, and Linux was just enough for me as a developer logging CLI, VIM, and tools like that. When I was in Dublin, Macbook Pro was the only available option; at InfluxData, I had a Thinkpad (the best laptop I ever had). Currently, I work on a Macbook Pro again because the non-apple available option was pretty low in terms of performance.</p>
<p>Now that you know my struggle for laptop and operating system consistency, a tool that works on both sounds appealing.</p>
<p>Nix has its own Linux distribution called NixOS. I slowly have a lot at it, but it is not a topic for this article.</p>
<h3 id="declarative-environment">Declarative environment</h3>
<p>The open-source project I mainly consistently from years is my <a href="https://github.com/gianarb/dotfiles">dotfiles</a> repository. I am probably the only person who knows how to run it, but it contains configuration for the various tools I use.</p>
<p>I have to admit that I would like to install it consistently and quickly on any of my on-demand servers I spin up, but I too lazy for it. Anyway, I like that approach because I describe what I want, and I can consistently get it everywhere. Nix gave me the same possibility, and it does not use a specification language like YAML, JSON, or whatever it uses a dialect.</p>
<p>It is a lazy, pure, and functional language. It is pretty awkward; I have to say, at least for my background. I didn’t figure it out yet, but the more I use it, and better it sticks to my mind.</p>
<p>I am also not that good when it comes to picking up new languages, it takes me some time, and I have to practice with them.</p>
<p>The good thing is that there are plantly of tutorials, each of them with different stakeholders. Do you like to be driven by example? There is <a href="https://nixos.wiki/wiki/Nix_Expression_Language">“Nix by example.”</a> You have time, and you want a more traditional <a href="https://nixos.org/manual/nix/stable/#ch-expression-language">reference manual</a>; they got you covered.</p>
<p>The fact that I don’t have to fight with the template engine makes me happy.</p>
<h3 id="it-is-all-based-on-git">It is all based on Git.</h3>
<p>I use Git since my first day at my first job. I was a solo developer, and my remote repository was not GitHub but a USB stick.</p>
<p>The Nix package manager is a GitHub repository. You can have your one, or you can use <a href="https://github.com/NixOS/nixpkgs">nixpkgs</a>. You can even merge multiple ones. Or import your derivation (the way Nix calls package).</p>
<p>A text editor and Git to clone a repository are what you need to look at all the packages, and their definition gives me a friendly feeling.</p>
<p>Based on how you want to define your environment, you can pin all the packages you are installing to a specific commit SHA from the package manager repository:</p>
<pre><code class="language-nix">let _pkgs = import <nixpkgs> { };
in
{ pkgs ?
import
(_pkgs.fetchFromGitHub {
owner = "NixOS";
repo = "nixpkgs";
#branch@date: nixpkgs-unstable@2021-01-25
rev = "ce7b327a52d1b82f82ae061754545b1c54b06c66";
sha256 = "1rc4if8nmy9lrig0ddihdwpzg2s8y36vf20hfywb8hph5hpsg4vj";
}) { }
}:
with pkgs;
</code></pre>
<p>Very powerful.</p>
<h3 id="environment-composition-with-nix">Environment composition with Nix</h3>
<p>I am not sure if environment composition has any sense, but it sounds descriptive to me. Nix is user and project aware.</p>
<p>With nix-env, you can install packages as a user. With nix-shell, you can manipulate your system at the project level. If you add NixOS to this chain, you get free customization at the operating system layer.</p>
<p>Currently, nix-shell is the tool I know more about, and I am in love with.</p>
<p>I didn’t experience those composition levels; I am currently writing my home-manager configuration file to solve my dotfiles repository’s required dependencies. Right now, I don’t have a way to install them automatically. I am not sure if that’s the right layer for such a problem yet, but I will figure it out soon.</p>
<h3 id="project-level-sandboxing-with-nix-shell">Project level sandboxing with nix-shell</h3>
<p>With a combination of symlinks and who knows what, nix-shell gives you a sandboxed environment with the only dependencies you need for your project. When you run nix-shell, it looks for a file called shell.nix that describes needed dependencies, environment variables, and so on. By default, you get all the commands and utilities you have in your system plus the one you declared for that project. If you have Go 1.15 in your system but want 1.13 for a single project nix-shell, you want to make it happen, for example. Tinkerbell has a <a href="https://github.com/tinkerbell/tink/blob/master/shell.nix">shell.nix</a> for almost all the repositories.</p>
<p>For some particular scenarios, I use the Docker container in development. But with Nix, I can remove that extra layer. I use containers and images to ship and run my applications on Kubernetes. Removing that layer decreases the need for volume mounting, port forwarding, the debugger works much more comfortably, and performance is the one your hardware provides to you, without virtualization if you are on Mac.</p>
<p>Containers in development are my way to go when for dependencies that I don’t care about or that I will never modify and have a state such as databases. But it is a joy to develop “locally.”</p>
<h3 id="everything-can-be-nixyfied">Everything can be “nixyfied”</h3>
<p>Passing the flag –pure to nix-shell won’t rely on the system installed packages but only on the one specified in nix-shell. It is a great way to validate that the declaration you wrote for your project can work everywhere you can run Nix. It makes continuous delivery what it should be a way to run workflows. It is not like that; for me, it is a constant translation exercise between Jenkinsfile, bash, YAML for GitHub Actions, drone, or Travis. With Nix, you declare the environment, and you can run it everywhere. For example, you can set shebang in your scripts, leaving to nix-shell the responsibility for satisfying the dependencies it needs:</p>
<pre><code class="language-sh">#!/usr/bin/env nix-shell
#!nix-shell -i bash ../shell.nix
make deploy
</code></pre>
<p>If you don’t want to translate from Nix to GitHub actions, there is an action that installs Nix, combined with the right shebang; you can reuse the shell.nix description for your project. I do that in <a href="https://github.com/gianarb/tinkie/blob/master/.github/workflows/ci.yaml">gianarb/tinkie</a>:</p>
<pre><code class="language-yaml">name: For each commit and PR
on:
push:
pull_request:
jobs:
validation:
runs-on: Ubuntu-20.04
env:
CGO_ENABLED: 0
steps:
- name: Checkout code
uses: actions/checkout@v2
- uses: cachix/install-nix-action@v12
with:
nix_path: nixpkgs=channel:nixos-unstable
- run: ./hack/build-and-deploy.sh
</code></pre>
<p>As you can see, I am not using actions to install the dependencies I need, I use <code>cachix/install-nix-action@v12</code> to get Nix, and everything is managed as I do locally. Something I don’t have to maintain, I suppose.</p>
<p>Mitchell Hashimoto uses Nix to provision its virtual machine, quickly enjoying the Linux environment when it comes to development compared with MacOS.</p>
<p class="text-center"><img src="/img/mitchellh-tweet-nixos.png" alt="Mitchel Hashimoto tweet: I switched my primary dev environment to a graphical NixOS VM on a macOS host. It has been wonderful. I can keep the great GUI ecosystem of macOS, but all dev tools are in a full screen VM. One person said “it basically replaced your terminal app” which is exactly how it feels." class="img-fluid" /></p>
<h3 id="conclusion">Conclusion</h3>
<p>I tend to avoid complications, and I am picky when it comes to the number of tools I have in my toolchain, but after a few months of observation, I think Nix deserves a place in my daily workflow. I just scratched the Nix surface; I didn’t even write my first derivation yet.</p>
<p>It brings back the joy I had a few years ago provisioning infrastructure, which I have lost in the last few years.</p>
<p class="small">Hero image via <a href="https://medium.com/@robinbb/what-is-nix-38375ed59484">Medium.com</a></p>
Evolution of a loglineThis is a story that represent the evolution I had in thinking and writing logs for an application. It highlights why they are important as a communication mechanism from your application and the outside. Explaining what I think are the responsibility we have as a developer when writing logs.https://gianarb.it/img/gianarb.png2021-01-15T10:08:27+00:002021-01-15T10:08:27+00:00https://gianarb.it/blog/evolution-of-a-logline<p>This story represents the evolution of a logline for myself when I write and
interpret it.</p>
<p>Late in 2012, I worked as a developer for a software agency in Turin,
specialized in software for tour operators. It was my second “real” job and the
first one not as a solo developer. Exciting time! An AJAX application with PHP
and MySQL backend running as a service developed mainly by a single person, the
lead developer. I interpreted the log back then as the equivalent of a save
point in a game. The tail was the primary tool to figure out what was going on;
a logline was helpful to figure out that a lot of customers were reaching a
particular line of code. The interpretation of the situation was up to humans.</p>
<p>Every developer involved in the project participates in adding the logline in
the codebase directly, developing the code, or indirectly when chatting about a
particular feature over lunch or doing a code review. Building a context from
an unknown log line was a useless exercise because the lead was always there
to help you figure out what that logline was supposed to tell.</p>
<p>Even the stream’s speed was a crucial metric to figure out the sanity of the
application. Where the logs too fast? The application was under heavy load, and
probably it was slow, not fast enough, or not smooth as usual, well something
was going on, and it was not good!</p>
<p>You can judge this story as unpractical but not as unusual. This approach does
not scale; it has an unmeasurable risk of “bus factor” but, if you don’t have a
panel in your Grafana dashboard representing the distribution of loglines, you
should look at it. Just for fun.</p>
<h1 id="bus-factor">Bus Factor</h1>
<p>Bus factor represents the risk of knowledge and responsibility centralization in
a single location. If the lead in my story resigns or gets hit by a bus, nobody
will build context from “not that descriptive” log lines quickly like him. And
the “speed of tail” requires to be very familiar with the stream. Sharing
knowledge and responsibility across the company, writing documentation, and
doing staff rotations are standard techniques that mitigate such risk.</p>
<h2 id="automation">Automation</h2>
<p>When your application state’s interpretation requires a human, it is tough to
build automation for it. Standardization in the way your application
communicates to the outside is another way to spread the knowledge in a team,
allowing you to write automation for it.</p>
<p>The format of a logline is the protocol to develop.</p>
<p>The format has to be parsable and useable by automation. You have to see logs as
a point in time, as time series more than as something that I should carefully
watch and try to interpret by myself.</p>
<p>A logline that looks like this:</p>
<pre><code>1610107485 New user inserted in the DB with ID=1234
</code></pre>
<p>Will become:</p>
<pre><code>time=1610107485 service="db" action="insert" id=1234 resource="user"
</code></pre>
<p>You can add a message that can be used to communicate with a person: msg=”new
user registered.” but not sure if it is mandatory, you can combine it later.</p>
<p>We do this exercise with ElasticSearch, applying full-text algorithms, and
tokenizing on the message. It is expensive, and it hides the developer’s
responsibility when it comes to consciously describe the current state of the
system with a logline. No, they are not random printf anymore. You can even see
it as JSON if you prefer.</p>
<p>Or, more in general, as a description of a particular point in time via
key-value pairs that you can aggregate, visualize, and use to drive powerful
automation, I work a lot in the cloud field. For me, reconciliation or a
system’s ability to repair itself is often based on those pieces of information.
If you want to go deeper on this topic, structured logging is what you should
look for.</p>
<h2 id="flexibility">Flexibility</h2>
<p>Having a logging library that allows you to do structured logging is a
must-have, and there is no answer about the number of key-value pairs you need.
The overall goal is to derive issues to learn the behavior of an application
from those points. It is not something that you use when having problems. The
application exposes the tool we have to figure out what a piece of code we
didn’t write is doing in production. In a highly distributed system, a logline
with the right fields such as hostname, PID, region, Git SHA, or version can
distinguish where the application having problems is running without looking
across many dashboards, Kubernetes UIs, and CLI.</p>
<p>Parsing and manipulating a structured log is more convenient than a random text
that has to be parsed and tokenized, but everything has a limit, so you have to
find the right balance based on experience. It is another never-ending iterative
process that we can call the evolution logline!</p>
<p><img src="/img/watch-4638673_1280.jpg" alt="This picture represents the time it takes to learn something new. It is a
picture of an open book with an old clock." class="img-fluid" /></p>
<h2 id="conclusion">Conclusion</h2>
<ul>
<li>Logline is the way you tech your application on how to communicate with the
outside.</li>
<li>Communication is useful in many fields. It is an opportunity to learn
something new or a way to communicate that we are in trouble. Same as logs,
use them as an opportunity to learn how a system works overall.</li>
<li>As a developer, do not see a logline as a random printf. The way it is
structured and articulated improves the communication quality between your
application the outside world.</li>
<li>A logline is not a fire and forget but an entity that evolves in time.</li>
<li>Logs represent the internal state of your application at some point in time
and somewhere in your codebase.</li>
</ul>
<p>Recently I spoke with <a href="https://twitter.com/lizthegrey">Liz</a> and
<a href="https://twitter.com/shelbyspees">Shelby</a> from HoneyComb about observability and
monitoring during
<a href="https://www.heavybit.com/library/podcasts/o11ycast/ep-32-managing-hardware-with-gianluca-arbezzano-of-equinix-metal/?utm_campaign=coschedule&utm_source=twitter&utm_medium=heavybit&utm_content=Ep.%20%2332,%20Managing%20Hardware%20with%20Gianluca%20Arbezzano%20of%20Equinix%20Metal">o11ycast</a>
a podcast about observability if you want to know more about this topic.</p>
<p class="small">Hero image via <a href="https://pixabay.com/illustrations/dna-string-biology-3d-1811955/">Pixabay</a></p>
Kubenetes v1.20 the docker deprecation dilemma in practicethere are many discussions going on Twitter about why Kubernetes v1.20 deprecated Docker and dockershim as default runtime. But it was a well thought an planned effort. Nothing to really worry about and here I will go over the process of updating Kubenetes from 1.19 to 1.20https://gianarb.it/img/gianarb.png2020-12-03T10:08:27+00:002020-12-03T10:08:27+00:00https://gianarb.it/blog/kubernetes-1-20-dockershim-in-practice<p>Kubernetes v1.20 is not yet out but there is already a lot going on behind the
scene. The main reason is the deprecation of Docker as default runtime.</p>
<p>I won’t go too deep in the theory because at this point I think it is a well
covered part. But a few things:</p>
<ol>
<li>If you run Docker, you run containerd. That’s it. Even if you didn’t know, or
you don’t like the idea.</li>
<li>The container runtime interface is there since a good amount of time and the
goal for it was to decouple the orchestrator (kubernetes), from other
business like running containers. Who cares about containers at the end.</li>
<li>Deprecating dockershim or at least removing it from the kubelet itself is the
right thing to do.</li>
</ol>
<p>More about this topic:</p>
<ul>
<li><a href="https://kubernetes.io/blog/2020/12/02/dockershim-faq/">Dockershim Deprecation FAQ</a></li>
<li><a href="https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/">Don’t Panic: Kubernetes and Docker</a></li>
</ul>
<p>I want to tell you how it works in practice. And this article contains my
experience updating a Kubernetes cluster from v1.19 to v1.20.</p>
<p>So I created a two node clusters on <a href="https://console.equinix.com/">Equinix
Metal</a> running Ubuntu using this simple script and
kubeadm.</p>
<pre><code class="language-bash">#!bin/bash
apt-get update
apt-get install -y vim git
apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
vim \
git \
software-properties-common
releaseName=$(lsb_release -cs)
if [ $releaseName == "groovy" ]
then
releaseName="focal"
fi
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
apt-key fingerprint 0EBFCD88
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
${releaseName} \
test"
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io
apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
</code></pre>
<p>I placed that script as CloudInit for my two nodes and I ran <code>kubeadm init/join</code>
to get my cluster.</p>
<pre><code class="language-terminal"># kubectl get node
NAME STATUS ROLES AGE VERSION
gianarb-k8s Ready master 2m11s v1.19.4
gianarb-k8s01 Ready <none> 91s v1.19.4
</code></pre>
<p>I have installed Flannel and now the nodes are ready. That’s it. That’s how I
measure success here.</p>
<p>It is not time to update to v1.20, so I downloaded the binaries from the
registry.</p>
<pre><code class="language-terminal"># wget https://dl.k8s.io/v1.20.0-rc.0/kubernetes-server-linux-amd64.tar.gz
# tar xzvf ./kubernetes-server-linux-amd64.tar.gz
# ./kubernetes/server/bin/kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.0-rc.0", GitCommit:"3321f00ed...
</code></pre>
<p>I checked available upgrade plan from <code>kubeadm</code> with the flag
<code>--allow-experimental-upgrades</code>, if you do this process when v1.20 will be
officially released, you won’t need that flag.</p>
<pre><code class="language-terminal">./kubernetes/server/bin/kubeadm upgrade plan --allow-experimental-upgrades
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks.
[upgrade] Running cluster health checks
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.19.4
[upgrade/versions] kubeadm version: v1.20.0-rc.0
[upgrade/versions] Latest stable version: v1.19.4
[upgrade/versions] Latest stable version: v1.19.4
[upgrade/versions] Latest version in the v1.19 series: v1.19.4
[upgrade/versions] Latest version in the v1.19 series: v1.19.4
I1203 21:50:13.860850 59152 version.go:251] remote version is much newer: ....
W1203 21:50:14.100325 59152 version.go:101] could not fetch a Kubernetes ....
W1203 21:50:14.100362 59152 version.go:102] falling back to the local client version: v1.20.0-rc.0
[upgrade/versions] Latest experimental version: v1.20.0-rc.0
[upgrade/versions] Latest experimental version: v1.20.0-rc.0
[upgrade/versions] Latest : v1.19.5-rc.0
[upgrade/versions] Latest : v1.19.5-rc.0
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
kubelet 2 x v1.19.4 v1.20.0-rc.0
Upgrade to the latest experimental version:
COMPONENT CURRENT AVAILABLE
kube-apiserver v1.19.4 v1.20.0-rc.0
kube-controller-manager v1.19.4 v1.20.0-rc.0
kube-scheduler v1.19.4 v1.20.0-rc.0
kube-proxy v1.19.4 v1.20.0-rc.0
CoreDNS 1.7.0 1.7.0
etcd 3.4.13-0 3.4.13-0
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.20.0-rc.0 --allow-release-candidate-upgrades
_____________________________________________________________________
The table below shows the current state of component configs as understood by this version of kubeadm.
Configs that have a "yes" mark in the "MANUAL UPGRADE REQUIRED" column require manual config upgrade or
resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually
upgrade to is denoted in the "PREFERRED VERSION" column.
API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED
kubeproxy.config.k8s.io v1alpha1 v1alpha1 no
kubelet.config.k8s.io v1beta1 v1beta1 no
_____________________________________________________________________
</code></pre>
<p>As you can see from the output there is <code>v1.20.0-rc.0</code> available so it is time
to apply that plan and rollout the upgrade.</p>
<pre><code class="language-terminal"># ./kubernetes/server/bin/kubeadm upgrade apply v1.20.0-rc.0 --allow-release-candidate-upgrades
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks.
[upgrade] Running cluster health checks
[upgrade/version] You have chosen to change the cluster version to "v1.20.0-rc.0"
[upgrade/versions] Cluster version: v1.19.4
[upgrade/versions] kubeadm version: v1.20.0-rc.0
[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y
...
</code></pre>
<p>Time to check kubelet with journalctl</p>
<pre><code class="language-terminal">#journalctl -xe -u kubelet
I1203 21:58:49.648539 108749 server.go:416] Version: v1.20.0-rc.0
I1203 21:58:49.649168 108749 server.go:837] Client rotation is on, will bootstrap in background
I1203 21:58:49.651975 108749 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
I1203 21:58:49.653428 108749 dynamic_cafile_content.go:167] Starting client-ca-bundle::/etc/kubernetes/pki/ca.crt
I1203 21:58:49.764200 108749 server.go:645] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /
I1203 21:58:49.764717 108749 container_manager_linux.go:274] container manager verified user specified cgroup-root exists: []
I1203 21:58:49.764743 108749 container_manager_linux.go:279] Creating Container Manager object based on Node Config: {Ru...
I1203 21:58:49.764862 108749 topology_manager.go:120] [topologymanager] Creating topology manager with none policy per container scope
I1203 21:58:49.764874 108749 container_manager_linux.go:310] [topologymanager] Initializing Topology Manager with none policy and container-level scope
I1203 21:58:49.764881 108749 container_manager_linux.go:315] Creating device plugin manager: true
W1203 21:58:49.765010 108749 kubelet.go:297] Using dockershim is deprecated, please consider using a full-fledged CRI implementation
</code></pre>
<p>We finally got it <code>Using dockershim is deprecated, please consider using a
full-fledged CRI implementation</code>. But everything still works.</p>
<pre><code class="language-terminal"># kubectl get node
NAME STATUS ROLES AGE VERSION
gianarb-k8s Ready control-plane,master 17m v1.20.0-rc.0
gianarb-k8s01 Ready <none> 16m v1.19.4
</code></pre>
<p>There is something cool as well, now the role is <code>control-plane</code> and <code>master</code>, I
am sure at some point we will deprecate <code>master</code> as well and this is just a
transition phase, same as it happened for Docker Shim.</p>
<p>We don’t want that working, so it is not time to change the default
configuration of <code>containerd</code>, because as you know, it is there sitting behind
docker since forever almost.</p>
<pre><code class="language-terminal">cat /etc/containerd/
# cat /etc/containerd/config.toml
...
# comment this line because we need cri enable
# disabled_plugins = ["cri"]
...
</code></pre>
<p>We need to enable the plugin <code>cri</code>, by default it is disabled when installing
docker-ce via the Docker registry because <code>dockerd</code> does not need a CRI, but
Kubernetes needs it obviously. Now you can restart the service with <code>systemctl</code>
and we have to tell the kubelet that now it has to use <code>containerd</code>.</p>
<pre><code class="language-terminal"># mkdir -p /etc/systemd/system/kubelet.service.d/
cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote
--runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
EOF
</code></pre>
<p>After a systemd daemon reload and a kubelet.service restart we are back again.</p>
<pre><code># kubectl get node
NAME STATUS ROLES AGE VERSION
gianarb-k8s Ready control-plane,master 23m v1.20.0-rc.0
gianarb-k8s01 Ready <none> 23m v1.19.4
</code></pre>
<p>Exercise for you, have a look at the kubelet logs, the warning is not there
anymore and you are good to go.</p>
Lessons learned working as site reliability engineerI want to share a few lessons I learned working three years as site reliability engineer. I kept the focus on the one I think are reusable, and that made me a better developer because reliability just as everything is an everybody businesshttps://gianarb.it/img/gianarb.png2020-11-26T10:08:27+00:002020-11-26T10:08:27+00:00https://gianarb.it/blog/lessons-learned-working-as-sre<p>I worked for three years at InfluxData as Site Reliability Engineer. When
onboarding a new role in this jungle called “career” in information technology,
you should be ready to learn what that role means.</p>
<p>Along the way, I developed new skills and mastered a few that I already had, but
they were not widely used elsewhere.</p>
<p>My title is not Site Reliability Engineer anymore, but the IT career is like a
roulette, and everything you learn will get back at some point. So I want to
share skills I think are essential when working as a Site Reliability Engineer
and that I feel they are useful to keep in your toolchain even when your title
changes.</p>
<h3 id="ability-to-develop-a-friendly-environment-for-yourself">Ability to develop a friendly environment for yourself</h3>
<p>One of my goals as a Site Reliability Engineer is to quickly support developers
having trouble with their code at scale.</p>
<p>Another one is to figure out criticalities when it comes to on-call.</p>
<p>I am far from my local environment in both cases, usually interacting with an
environment much more complicated. Having something you can call familiar help.
It can be whatever:</p>
<ul>
<li>A few bash scripts that wrap other commands with a UX hard to remember</li>
<li>A CLI tool you wrote with your team</li>
<li>A directory where you can quickly go and write notes about what is going on
for future use</li>
<li>A set of bullet points or a runbook you know is rock solid and can drive you
where you want to go.</li>
</ul>
<p>Those are just a few tricks I use. If you have yours and you want to share them
as a comment, please do it!</p>
<p>This is an essential skill that everyone has to master, but as an SRE, when you
have to act quickly, I really learned it, and now I do my best to develop my
workflows and a working environment I like. Those days I am giving a try with
Nix and, in particular, nix-shell because it helps me customize my environment
without the overhead of a Docker container.</p>
<p>This may sound time-consuming. Many projects have a README describing tools and
requirements to contribute or build the project. Why I need my way? Well, I am
not saying you should start from zero, but when I glue it with the flavor I
like, I code better and am happier. So for me, it is a big YES!</p>
<h3 id="troubleshoot-like-a-ninja">Troubleshoot like a ninja</h3>
<p>Starting from the same purpose as before, a Site Reliability Engineer looks at
the code when it runs in production, and production is a scary and dangerous
place. As a developer, if you are lucky and smart, you try to focus on one
application at a time, yes it probably has many dependencies, but still, code
moves one line at a time.</p>
<p>In production with concurrency and thousands of requests happening almost
simultaneously, things get pretty messy. Having the ability and the right tools
to slice and dice from different points of view and prospectives, from an entire
region to a specific application requires operational experience and training.</p>
<p>A good exercise is to have the desire to troubleshoot everything. Does a
teammate have a question about a system in production? Go and help him. Visit
logs, traces, and dashboards even when everything looks quiet if you are so
lucky to have a definition of it.</p>
<p>Another thing I do, but more in general, is to follow the best. There is a lot
about the topic in the forms of books, talks, and similar. Read them but even
more importantly, follow who master these topics every day:
<a href="https://twitter.com/rakyll">@rakyll</a>,
<a href="https://twitter.com/brendangregg">@brendangregg</a>,
<a href="https://twitter.com/relix42">@relix42</a>,
<a href="https://twitter.com/lizthegrey">@lizthegrey</a>,
<a href="https://twitter.com/lauralifts">@lauralifts</a>. Please do not follow them on
Twitter only, but look at their GitHub as well; sometimes, a small project that
works reliably and well for us is gold.</p>
<h3 id="think-about-code-debuggability-in-production">Think about code debuggability in production</h3>
<p>Like everything you read so far, it is always essential because, as I said, Site
Reliability Engineer is just a role that has a subset of responsibilities and
objectives, but we are not silos; everything matters a feature has to be usable;
good looking, functional. Code review for me become a lot more about: “is this
code understandable in production?”, “what do I want it to tell me when running
at scale?”, “how this trace looks like?” “is this log useful, and how does it
impact the overall context?”.</p>
<p>Those questions strongly show my mind when working as an SRE, but they made me a
better developer. I still try to answer them when coding or when doing code
reviews.</p>
<h3 id="conclusion">Conclusion</h3>
<p>Titles are just titles; reading them will help to know a set of skills you
leveraged most, but that’s it, and it is not always true. You are not married to
your title, and if you are curious about various aspects of our work, you will
change many of them. The right balance of all those skills will make you unique.</p>
Vanity URL for Go mod with zero infrastructureA lot of Go modules today are hosted on GitHub. But you can setup vanity URL using a custom domain. This is good to decouple your library from GitHub. The best part is that it does not require any infrastructure, if you don't want to.https://gianarb.it/img/gianarb.png2020-11-13T10:08:27+00:002020-11-13T10:08:27+00:00https://gianarb.it/blog/go-mod-vanity-url<p>This post is about how I renamed a go module from
github.com/something/somethingelse to go.gianarb.it/somethingelse. It requires
zero infrastructure, just a static site that can run on GitHub.</p>
<p>I used one of the many projects I have that nobody cares about
<a href="https://github.com/gianarb/go-irc">gianarb/go-irc</a>.</p>
<h3 id="why">Why</h3>
<p>You know, is one of those ideas you have in your mind for ages, but who cares?
At least for me that I don’t have any cool open source project under my name.</p>
<p>I am not one of those people who suffer when realizing that it is not escale. If
you end up having a project that gets traction and is tight to github.com
because you didn’t think about another way to go, you are stuck. And even if
GitHub today is cool, it won’t stay cool forever.</p>
<p>Filippo Valsorda <a href="https://twitter.com/FiloSottile">@FiloSottile</a> today
<a href="https://twitter.com/FiloSottile/status/1327240411266641920">tweeted</a> about this
topic, and I looked at how he set up filippo.io/age to solve this little
dilemma.</p>
<h3 id="goals">Goals</h3>
<p>This is not about how to escape from GitHub but is about setting up a “vanity
URL” that won’t lock you or your project to GitHub. It does not require any
infrastructure, just a domain that you can point to a GitHub Pages.</p>
<h3 id="prerequisite">Prerequisite</h3>
<ol>
<li>Create a DNS record that points as CNAME to <code><github-handle>.github.io</code>. I
used go.gianarb.it</li>
<li>Create a repository; it will be the home for your static site. Mine is
<a href="https://github.com/gianarb/go-libraries">gianarb/go-libraries</a></li>
<li>Set the repository up to be a <a href="https://pages.github.com/">GitHub page</a> and
enable HTTPS. You can enable it via the repository Settings; we will push
HTML files to it directly, so I used the master branch as the GitHub page’s
source.</li>
</ol>
<h3 id="add-your-first-library">Add your first library.</h3>
<p>If your library is already using go mod, you have to change the module name to
the new one. In my case, from was github.com/gianarb/go-irc to
go.gianarb.it/irc. I just searched and replaced with my editor in all the
project. Renaming a module is a bc break; I am not sure how to avoid or mitigate
that; if you know, let me know!</p>
<p>You can push a new file to your static site repository; I called my irc:</p>
<pre><code class="language-html"><html>
<head>
<meta name="go-import" content="go.gianarb.it/irc git https://github.com/gianarb/go-irc">
<meta http-equiv="refresh" content="0;URL='https://github.com/gianarb/go-irc'">
</head>
<body>
Redirecting you to the <a href="https://github.com/gianarb/go-irc">project page</a>...
</body>
</html>
</code></pre>
<p>Replace your URL accordingly, but as soon as you push this file and GitHub will
release it to your page, you will be able to import: <code>go.gianarb.it/irc</code>.</p>
<h3 id="conclusion">Conclusion</h3>
<p>This methods works as a safeguard if you decide to move your code out from
GitHub. The static site can be deployed to Netlify, S3 or served by Nginx. It
does not need to stay on GitHub.</p>
<p>Same for your code, if you decide to move from GitHub to GitLab you can do it
transparently.</p>
What is Tinkerbell?I want to share with you what is Tinkerbell. An open source project I help maintaining, developed by Equinix Metal. Tinkerbell helps you to manage your hardware and datacenter programmatically via an API.https://gianarb.it/img/gianarb.png2020-11-06T10:08:27+00:002020-11-06T10:08:27+00:00https://gianarb.it/blog/what-is-tinkerbell<p>First things first, Tinkerbell is an open-source project mainly written in Go
that comes from PacketHost, now Equinix Metal. Equinix Metal is a cloud provider
that serves bare metal servers. No virtual machines, no high-level services, I
said bare metal! Imagine a colocation that you can rent per hour.</p>
<p>Tinkerbell is the software Equinix Metal dreamed about as an internal
provisioner for datacenter automation. They took their internal provisioner and
removed any PacketHost references of business specific code, and pushed it to
GitHub for the community to enjoy the same technologies.</p>
<p>The project is a number of micro-services that provide various functionality to
configure hardware and provision bother Operating System and additional software
through it’s workflow engine.</p>
<h2 id="what-the-project-provides">What the project provides</h2>
<ul>
<li>The first micro-service is called
<a href="https://github.com/tinkerbell/boots">boots</a>. This Tinkerbell service
provides a DHCP and a TFTP server to tell a piece of hardware (server) what
to do when netbooting, it provides this information through the iPXE
project.</li>
<li>Tinkerbell serves a CLI that you can use to interact with a control plane
that serves HTTP and gRPC API. The service which does all those things is in
the <a href="https://github.com/tinkerbell/tink">tink</a> repository, and it provides
three binaries: tink-server (the control plane), tink-CLI, the command line
interface, and tink-worker.</li>
<li>Tinkerbell provides an operating system that runs in memory, it is based on
Alpine, and it is called <a href="https://github.com/tinkerbell/osie">Osie</a>. This
in-memory operating system runs directly on the hardware you want to
provision, and it runs the tink-worker.</li>
<li>Once the in-memory Osie has started it begins the tink-worker which in turn
communicates with the control plane (tink-server), asking for any work that
has to be done on that server. This unit of work is called a workflow.</li>
<li><a href="https://github.com/tinkebell/hegel">Hegel</a> is a metadata server, comparable
to the AWS EC2 metadata or the Equinix Metal one; the majority of cloud
vendors provide this type of service , so you should have it as well! It is
crucial when running scripts in a particular server because you can get
concrete variables from it, such as the operating system it runs, its IPs,
location, and so on.</li>
</ul>
<h2 id="the-end-goal">The end goal</h2>
<p>The Tinkerbell end work is to bring to life a piece of hardware.</p>
<h2 id="workflow-and-template">Workflow and template</h2>
<p>A template is a specification file that describes what we want to execute. A
workflow starts from a template, and it has a particular target. Templates are
reusable; workflows are a single execution and can’t be reused. The single unit
of work in a template is called action. You can get as many actions you want in
a template, and each action runs in its own Docker container.</p>
<h2 id="action">Action</h2>
<p>As mentioned above, actions are Docker containers and that means that you can
build each action in isolation in the language you want. It can use python,
bash, Golang, Rust, or whatever you can run in a container.</p>
<p>You may think that Docker would sound like an overhead, however we took a
natural decision based on how we could use the container concept in operations.
The concept of build, pull, and push has become commonplace within development
environments, and we think it could also work well in operational environments
too. Building containers to contain operational tasks in isolation and enhancing
that with testing and simplified execution of a container is a clear benefit. It
is an effective way to move code around in a reusable way without having to
reinvent the distribution model. Some of the actions you will see very often in
a Tinkerbell workflow may be:</p>
<ul>
<li>Disk related actions: mounting a disk, wiping it, or setting up a partition
table to boot an operating system</li>
<li>Downloading an Operating System like Ubuntu, Debian, NixOS, CentOS</li>
<li>Copy an operating system in a partition</li>
</ul>
<p>But you will be able to write actions related to your business:</p>
<ul>
<li>notify a particular API when provisioning fails</li>
<li>Attempt a recovery</li>
<li>Observe and mark the status of your provisioning</li>
<li>Who knows! There are no limitations here.</li>
</ul>
<h2 id="how-a-template-and-a-workflow-looks-like">How a template and a workflow looks like</h2>
<p>Unfortunately, there are not many examples, but as maintainers, the next three
months will be all about public workflows and reusable actions.</p>
<p>Kinvolk wrote a blog post about <a href="https://kinvolk.io/blog/2020/10/provisioning-flatcar-container-linux-with-tinkerbell/">how to provision
Flatcar</a>
on bare metal with Tinkerbell.</p>
<p>The Tinkerbell documentation <a href="https://tinkerbell.org/examples/hello-world/">has an
example</a> of a “hello world.”
template.</p>
<p><a href="https://www.fransvanberckel.nl/">Frans van Berckel</a> wrote a workflow for
<a href="https://github.com/fransvanberckel/debian-workflow">CentOS</a> and
<a href="https://github.com/fransvanberckel/debian-workflow">Debian</a>.</p>
<p>One of my next projects will be to write a workflow that won’t install an
operating system. It will start something like k3s or k8s directly on Osie for
my ephemeral homelab! I am not sure it has a sense or will ever work, but I
think it is an excellent example: “it is not all about having a persisted and
traditional operating system those days.”</p>
<h2 id="how-to-get-started">How to get started</h2>
<p>We put a fair amount of effort into a
<a href="https://github.com/tinkerbell/sandbox">sandbox</a> project and setup guide. You
can run it <a href="https://tinkerbell.org/docs/setup/local-with-vagrant/">locally with Vagrant
</a>or on <a href="https://tinkerbell.org/docs/setup/terraform/">Equinix
Metal</a>.</p>
<p>Aaron Ramblings wrote a blog post, <a href="https://geekgonecrazy.com/2020/09/07/tinkerbell-or-ipxe-boot-on-ovh/">“Tinkerbell or iPXE boot on
OVH”</a>
using the sandbox to run Tinkerbell on OVH! I am still surprised when I read it
because he experimented with the sandbox in a very early stage of the project,
and in the same way, he was able to run sandbox on OVH; it can run almost
wherever else (at least for the control plane part).</p>
<h2 id="next-steps">Next steps</h2>
<p>With the help of our community we recently improved our continuous integration
pipeline to build all the projects for various architecture: <code>linux/386</code>,
<code>linux/amd64</code>,<code> linux/arm/v6</code>, <code>linux/arm/v7</code>, <code>linux/arm64</code> levering Docker
buildx, Qemu, and GitHub Actions. My goal was to be able to run the provisioner
in a Raspberry Pi. Because as I wrote before, my homelab tends to go away, get
moved, disconnected, and I think I can keep running reliably only a Raspberry PI
as it is today. So I want to run the control plane on a RaspberryPI. I presume
there are smarter things to do with multi-arch, but let’s be honest; we all have
a RaspberryPI leftover somewhere.</p>
<p>We use the sandbox project as a way to release Tinkerbell’s version as an all
project. We are pinning all the various dependencies such as Boots, Hegel,
Tink-Server, CLI, Osie, and when they all pass the integration tests, we tag a
new release. The generated artifacts are containers for now. We want to get
binaries in this way. You can run Tinkerbell as you like, even without
containers. At some point, we will tag and manage each component independently,
but for now, it is a lot of effort.</p>
<p>Releasing new workflows is something we are working on already. So stay tuned!</p>
<p>Another project is available in the Tinkerbell GitHub organization that I didn’t
mention because it is not hooked yet as part of the stack. After all, we are
working at its version two. <a href="https://github.com/tinkerbell/pbnj">PBNJ</a> provides
a standard API to interact with various BMCs and IPMIs (Intelligent Platform
Management Interface). Having this kind of ability in a datacenter is essential
because we want to pilot things like reboot, restart, switch off for each server
programmatically, and even as part of a workflow.</p>
<h2 id="conclusion">Conclusion</h2>
<p>There already exists huge demand for bare-metal usage, which with the growth
caused by things like 5G, dedicated GPUs/FPGAs, HPC, constant and expected
performance and security boundaries is only going to grow. A recent report by
the <a href="https://www.mordorintelligence.com/industry-reports/bare-metal-cloud-market">Mordor Intelligence
company</a>
reports “The bare metal cloud market was valued at USD 1.75 billion in 2019 and
expected to reach USD 10.56 billion by 2025” which clearly shows that there is
growing demand for a modern platform to provision their bare-metal
infrastructure.</p>
<p>Datacenter management is hard, and that’s why the public cloud got so much
traction. For companies and products, managing hardware is unnecessary and a
distraction, but when it becomes a requirement or when you think it is strategic
to manage your own hardware Tinkerbell and its community comes to rescue you.</p>
<p class="alert alert-info">A big thank you goes to <a href="https://twitter.com/thebsdbox">Dan</a> for his review and
support writing this article!</p>
<h2 id="more-i-want-more">More, I want more!</h2>
<p><a href="https://www.youtube.com/watch?v=Y04eCSKaQCc">Dan and Jeremy had a conversation</a>
about netbooting and bare metal provisioning. It is available on YouTube, you
should really have a look at it!</p>
<p><a href="https://www.youtube.com/watch?v=QxpKnMGywTU">Alex Ellis and Mark Coleman recorded a
video</a> setting up and using Tinkerbell.
The video is a bit out of date and they did not use the new sandbox project
because it was not available at that time. But still a good and valuable!</p>
Reactive planing in Golang. Reach a desired number adding and subtracting random numbersAn example about how to write reactive planning in Go. Code and step by step solution for an exercise I developed to learn plannerhttps://gianarb.it/img/gianarb.png2020-10-26T10:08:27+00:002020-10-26T10:08:27+00:00https://gianarb.it/blog/reactive-plan-golang-example<p>Ciao! A few months ago, probably a year, I wrote a small library called
<a href="https://github.com/gianarb/planner">planner</a>. It comes from my experience using
reactive planning and Kubernetes. I am really in love with this way of writing
code because it sounds very reliable to me.</p>
<p>Over the last couple of days, I decided to write documentation for it! So now it
is presentable; I streamed that with <a href="https://twitch.tv/gianarb">Twitch</a> if you
like to watch people coding!</p>
<p>As part of the library’s readme, I wrote a small program, and I left a couple of
exercises to the reader. With this article, I want to solve them.</p>
<p>You can follow this article and try it yourself, starting from
<a href="https://play.golang.com/p/0LuIoMtp10f">play.golang.com</a>.</p>
<pre><code class="language-golang">package main
import (
"context"
"time"
"github.com/gianarb/planner"
"go.uber.org/zap"
)
func main() {
ctx, done := context.WithTimeout(context.Background(), 10*time.Second)
defer done()
countPlan := &CountPlan{
Target: 20,
}
scheduler := planner.NewScheduler()
scheduler.WithLogger(initLogger())
scheduler.Execute(ctx, countPlan)
}
type CountPlan struct {
Target int
current int
}
func (p *CountPlan) Create(ctx context.Context) ([]planner.Procedure, error) {
if p.current < p.Target {
return []planner.Procedure{&AddNumber{plan: p}}, nil
}
return nil, nil
}
func (p *CountPlan) Name() string {
return "count_plan"
}
type AddNumber struct {
plan *CountPlan
}
func (a *AddNumber) Name() string {
return "add_number"
}
func (a *AddNumber) Do(ctx context.Context) ([]planner.Procedure, error) {
a.plan.current = a.plan.current + 1
return nil, nil
}
func initLogger() *zap.Logger {
cfg := zap.NewProductionConfig()
cfg.Encoding = "console"
l, _ := cfg.Build()
return l
}
</code></pre>
<p>This program tries to reach the <code>Target</code> (20 in this case) adding numbers to the
current state. If you execute this program as it is you will get the following
logs:</p>
<pre><code class="language-console">1.257894e+09 info planner@v0.0.1/scheduer.go:41 Started execution plan count_plan {"execution_id": "98d28eed-9b3b-4ad8-bfbd-1b5338d1a649"}
1.257894e+09 info planner@v0.0.1/scheduer.go:59 Plan executed without errors. {"execution_id": "98d28eed-9b3b-4ad8-bfbd-1b5338d1a649", "execution_time": "0s", "step_executed": 20}
</code></pre>
<p>As you can see, the scheduler executed the plan <code>count_plan</code> successfully, and
it took 20 steps to get there (<code>step_executed: 20</code>).</p>
<p>Reasonable because, as you can see, the <code>CounterPlan.Create</code> function returns an
<code>AddNumber</code> procedure and that procedure only adds 1 to the current state. It is
just a counter; let’s make it a bit more fun. I want to add or substract random
number until the target is reached. The program adds when the current state is above the
target, when above it subtracts. If it’s equal we are done. This is a simple way
to simulate something that has to adapt, too simple to sound cool but still
something understandable.</p>
<h3 id="change-the-addnumber-to-use-a-randomly-generated-number">Change the AddNumber to use a randomly generated number.</h3>
<p>We need to change the AddNumber in order to add not 1 but a random number. Let’s
do it:</p>
<pre><code class="language-go">var random *rand.Rand
func initRandom() {
s1 := rand.NewSource(time.Now().UnixNano())
random = rand.New(s1)
}
</code></pre>
<p>At this point, we can use <code>random</code> as part of the <code>AddNumber.Do</code> function.</p>
<pre><code class="language-go">type AddNumber struct {
plan *CountPlan
}
func (a *AddNumber) Name() string {
return "add_number"
}
func (a *AddNumber) Do(ctx context.Context) ([]planner.Procedure, error) {
a.plan.current = a.plan.current + random.Intn(10)
return nil, nil
}
</code></pre>
<p>For simplicity, I am taking a random number between 0 and 10. What happens now?
The problem now is that we can go above the target, so we have to make our
<code>CounterPlan.Create</code> function and our logic a bit more complicated.</p>
<h2 id="evolve-the-create-function-to-subtract-numbers-from-the-current-state">Evolve the Create function to subtract numbers from the current state</h2>
<pre><code class="language-go">func (p *CountPlan) Create(ctx context.Context) ([]planner.Procedure, error) {
if p.current < p.Target {
return []planner.Procedure{&AddNumber{plan: p}}, nil
} else if p.current > p.Target {
return []planner.Procedure{&SubtractNumber{plan: p}}, nil
}
return nil, nil
}
</code></pre>
<p>When we go above the target, the Plan subtracts a random number, and it keeps
going until we get to it. <code>SubtractNumber</code> does the opposite of what
<code>AddNumber</code> does, it subtracts a random number between 0 an 10.</p>
<pre><code class="language-go">type SubtractNumber struct {
plan *CountPlan
}
func (a *SubtractNumber) Name() string {
return "subtract_number"
}
func (a *SubtractNumber) Do(ctx context.Context) ([]planner.Procedure, error) {
a.plan.current = a.plan.current - random.Intn(10)
return nil, nil
}
</code></pre>
<p>You can run the result <a href="https://play.golang.com/p/JDuizzUI86M">here</a>, and you
will see that based on the random numbers, it adds or subtracts the number of
executed steps changes.</p>
<p>NOTE: the golang playground always starts from the same time; in my example, I
use time as Seed; for this reason, to see a variation in the number of steps,
you will have to run the code locally.</p>
<h2 id="conclusion">Conclusion</h2>
<p>This is probably a too straightforward example, but let’s imagine that
your Target is not fixed and varies based on external factors. Your house’s
temperature and this program is a thermostat that has to keep your room at the
desired temperature. Or the number of instances running in your cloud provider,
and you have to keep them balanced. This last use case is the exact problem I
solved writing <a href="https://github.com/gianarb/keepit">keepit</a> a replica set for
<a href="https://metal.equinix.com">Equinix Metal</a> servers. I used planner, so check it out.</p>
<p>I didn’t highlight this example because this pattern gives you an excellent way
to measure how reliable your program is. Think about it in this way; you can
programmatically handle errors returning a procedure or more than one that can
mitigate the error itself. It can be a “sleep for 5 minutes and retry”, or you
can do something more complicated, and until the Plan keeps returning work to
do, you will have the opportunity to succeed. I extracted an highlight from the
<a href="https://www.twitch.tv/videos/780401570">Twitch stream</a> rambling about this.</p>
<p>Have a nice week!</p>
Your release workflow is code, it is just about timeI think code is some way has to win against specification languages or DSL or even from languages that are not easy to move around like bash. The Kubernetes sig-release is migrating a bunch of scripts from bash to Go and I think it is the right way to go. You will do the same, it is just about time.https://gianarb.it/img/gianarb.png2020-10-21T10:08:27+00:002020-10-21T10:08:27+00:00https://gianarb.it/blog/release-workflow-as-code<p>20th October 2020 is the day I released Kubernetes for the first time. To be
precise, I piloted with the help of sig-release Kubernetes <code>v1.20.0alpha3</code>. One of
the reasons I am happy to work with the sig-release is to learn and see how such
a significant process gets released reliably and continuously by a group of
people coming from different backgrounds, jobs, and locations.</p>
<p>The first lesson learned you would notice as soon as you join the SIG meeting
more frequently or as soon as you start contributing is the general effort in
converting what used to be bash scripts to Go.</p>
<p>Now, don’t fight against the languages by themself, but I think the story is
reasonable. You start small, and when it comes to releasing code, a lot happens
in somebody’s terminal. That’s why many of the release workflows I saw in my
life are a mix of Makefile and bash script.</p>
<p>I don’t think it scales because it is hard to get error handling, retry logic,
and testing made right in bash. Maybe I am just not good enough with BASH, and I
know there are testing libraries for it like
<a href="https://github.com/sstephenson/bats">bats</a>, for example.</p>
<p>Anyway, I have to admit, I feel good enough with BASH, but I code way better in
Go, PHP, and probably even JavaScript. Also, I am sure this is a feeling a share
with many people, and more in general, the Kubernetes development community is
very fluent with Golang.</p>
<p>Anyway, let’s treat the code that empowers the release lifecycle as application
code, just as the Sig Release is doing with Kubernetes. Documentation, testing,
user experience, and so on. Develop useful libraries that can be encapsulated in
command-line tools, or API, or bots.</p>
<p>There is a BASH script that takes snapshots from
<a href="http://testgrid.k8s.io/">Testgrid</a> called
<a href="https://github.com/kubernetes/release/blob/master/testgridshot">testgridshot</a>,
uploads them to Google Cloud, and outputs a markdown that can be copy-pasted as
a comment in <a href="https://github.com/kubernetes/sig-release/issues/1296">the issue we use to track every
release</a>. We run it to
take a snapshot of the various testing pipeline status at the time of a release.</p>
<p>testgridshot is the unique one in BASH I had to interact with for now, and it
didn’t work because of some environmental issues with my laptop. Coincidence? It
can be solved by running it as a container and having a binary with statically
compiled with all the needed dependencies.</p>
<p><a href="https://twitter.com/comedordexis">Carlos</a> is currently working on rewriting
testgridshot in Golang; it will use as a command-line interface, and I think it
will be even better to encapsulate it as prow capability.</p>
<p><a href="https://github.com/kubernetes/test-infra/tree/master/prow">Prow</a> is the
Kubernetes CI/CD system. It can trigger jobs for particular actions, and almost
everything you see happening in GitHub when using <code>/</code> commands like <code>/open</code> <code>/assign</code>
and so on is a Prow responsibility.</p>
<p>Testgridshot is useful during a release cut. The cut starts from a GitHub issue;
as we saw, it sounds very comfortable to have a command available like
/testgridshot and leaving Prow the responsibility to comment.</p>
<p>Now the takeaway hidden by the word <strong>encapsulates</strong>, it is great to have both a
CLI and a Prew command. Go becomes your baseline where the operational
experience live. All the rest is a UX, and you can have as many of them.</p>
<p>I am not writing this because you should stop and move all your BASH to
something else, but I experienced by myself. I see this little story with the
Kubernetes SIG release to confirm that it’s easy to block ourselves as release
engineering because there is a BASH script that we don’t want to rewrite. After
all, it is like that since forever. The project is not the same since day one,
the team grew or changed, and it is reasonable for a workflow to follow this
evolution.</p>
How bare metal provisioning works in theoryWhat I learned about how bare metal provisioning works developing tinkerbell.https://gianarb.it/img/gianarb.png2020-10-08T10:08:27+00:002020-10-08T10:08:27+00:00https://gianarb.it/blog/how-bare-metal-works-in-theory<p>I am sure you heard about bare metal. Clouds are made of bare metal, for example.</p>
<p>The art of bringing to life an unanimated piece of metal like a server to something useful is something I am learning since I joined <a href="https://metal.equinix.com">Equinix Metal</a> in May.</p>
<p>Let me make a comparison with something you are probably familiar with. Do you know why Kubernetes is hard? Because there is not one Kubernetes. It is a glue of an unknown number of pieces working together to help you deploy your application.</p>
<p>Bare Metal is almost the same, hundreds of different providers, server size, architectures, chips that in some way you have to bring to life.</p>
<p>We have to work with some common concepts we have. When a server boots, it runs a BIOS that looks in different places for something to run:</p>
<ol>
<li>It looks for a hard drive</li>
<li>It looks for external storage like a USB stick or a CD-Rom</li>
<li>It looks for help from your network (netbooting)</li>
</ol>
<p>Options one and two are not realistic if the end goal is to get to a handsfree, reliable solution. I am sure cloud providers do not have people running around with a USB stick containing operating systems and firmware.</p>
<h2 id="netbooting">Netbooting</h2>
<p>I spoke about <a href="https://gianarb.it/blog/first-journeys-with-netboot-ipxe">my first experience netbooting Ubuntu</a> on my blog. That article is efficient with reproducible code. Here the theory.</p>
<p>When it comes to netbooting, you have to know what PXE means. Preboot Execution Environment is a standardized client/server environment that boots when no operating system is found, and it helps an administrator boot an operating system remotely. Don’t think about this OS as the one you have in your laptop, I mean, technically it is, but the one your run there or in a server is persisted, that’s why you can have files that survive a reboot.</p>
<p>The one you start with PXE runs in memory, and from there, you have to figure out how to get the persisted OS you will run in your machine.</p>
<p>When the in-memory operation system is up and running, you can do everything you are capable of with Ubuntu, Alpine, CentOS, or Debian. In practice, what people tend to do is to run applications and scripts to format a disk with the right partition and to install the end operation system.</p>
<p>Pretty cool. PXE is kind of old, and for that reason, it is burned into a lot of different NICs. You will hear a lot more about iPXE, a “new” PXE implementation. What is cool about those is the <code>chain</code> function. From one PXE/iPXE environment, you can chain another PXE/iPXE environment. That’s how you get from PXE (that usually runs by default in a lot of hardware (if you have a NUC you run it)) to iPXE.</p>
<pre><code>chain --autofree https://boot.netboot.xyz/ipxe/netboot.xyz.lkrn
</code></pre>
<p>iPXE supports a lot more protocols usable to download OS from such as TFTP, FTP, HTTP/S, NFC…</p>
<p>This is an example of iPXE script:</p>
<pre><code>#!ipxe
dhcp net0
set base-url http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/legacy-images/netboot/ubuntu-installer/amd64/
kernel ${base-url}/linux console=ttyS1,115200n8
initrd ${base-url}/initrd.gz
boot
</code></pre>
<p>The first command, <code>dhcp net0</code>, gets an IP for your hardware from the DHCP. <code>kernel</code> and <code>initrd</code> set the kernel and the initial ramdisk to run in memory.</p>
<p><code>boot</code> starts the <code>kernel</code> and the <code>initrd</code> you just set.</p>
<p>There is more; this is what I find myself using more often.</p>
<h3 id="infrastructure">Infrastructure</h3>
<p>To netboot successfully, you need to distribute a couple of things:</p>
<ol>
<li>An iPXE script</li>
<li>The operating system you want to run (kernel and initrd)</li>
</ol>
<h3 id="workflow">Workflow</h3>
<ol>
<li>Server starts</li>
<li>There is nothing to boot in the HD</li>
<li>Starts netbooting</li>
<li>It makes a DHCP request to get network configuration, and the DHCP returns the TFTP address with the location of the iPXE binary</li>
<li>iPXE starts and makes another DHCP request; the response contains the URL of the iPXE scripts with the commands you saw above</li>
<li>At this point, iPXE runs the script, downloads the kernel, and the initrd with the protocol you specified, and it runs the in-memory operating system.</li>
</ol>
<p>Pretty cool!</p>
<h2 id="the-in-memory-operating-system">The in-memory operating system</h2>
<p>The in-memory operating system can be as smart as you like; you can build your one, for example, starting from Ubuntu or Alpine. Size counts here because it has to fit in memory.</p>
<p>When the operating system starts, it runs as PID 1, what is called <code>init.</code> It is an executable located in the ramdisk called <code>/init.</code> That script can be as complicated as you like. It can be a problematic binary that downloads from a centralized location commands to execute, or it can be bash scripts that format the local disk and installs the final operating system.</p>
<p>What I am trying to say is that you have to make the in-memory operating system useful for your purpose. If you use native Alpine or Ubuntu, the init script will start a bash shell, not that useful.</p>
<h2 id="dhcp">DHCP</h2>
<p>As you saw, the DHCP plays an important role. It is the first point of contact between unanimated hardware and the world. If you can control what the DHCP can do, you can, for example, register and monitor the healthiness of a server.</p>
<p>Imagine you are at your laptop, and you are expecting a hundred new servers in one of your datacenters, monitoring the DHCP requests. You will know when they are plugged into the network.</p>
<h2 id="containers-what">Containers what?</h2>
<p>Containers are a comfortable way to distribute and run applications without having to know how to run them. Think about this scenario. Your in-memory operating system at boot runs Docker. The <code>init</code> script at this point can pull and run a Docker container with your logic for partitioning the disk and installing an operating system, or it runs some workload and exit leaving space for the next boot (a bit like serverless, but with servers, way cooler).</p>
<p>Or the Docker Container can run a more complex application that reaches a centralized server that dispatches a list of actions to execute via a REST or gRPC API. Those actions can be declared and stored from you.</p>
<h2 id="conclusion">Conclusion</h2>
<p>The chain of tools and interactions to get from a piece of metal to something that runs some workload is not that long. Controlling all the steps and the tools along the way gives provides the ability to provisioning cold servers from zero to something that developers know better how to use.</p>
<p>Ok, I lied to you. This is not just theory. This is how <a href="https://tinkerbell.org">Tinkerbell</a> works.</p>
<p class="small">This post was originally posted on <a href="https://dev.to/gianarb/how-bare-metal-provisioning-works-in-theory-1e4e">dev.to</a>.</p>
Thinking in Systems written by Donella H. MeadowsReview of the book Thinking in Systems written by Donella Meadowshttps://gianarb.it/img/thinking-in-systems-book.jpg2020-09-12T10:08:27+00:002020-09-12T10:08:27+00:00https://gianarb.it/blog/thinking-in-systems-donella-meadows-review<p><a href="https://amzn.to/3khu7k9">“Thinking in Systems”</a> acted on me as a <strong>reinforcing loop</strong>. The motivation I have to
explore the ability to think of Software as loops and systems show up reading
those pages.</p>
<blockquote>
<p>The second kind of feedback loop is amplifying, reinforcing, self-multiplying,
snowballing– a vicious or virtuous circle that can cause healthy growth or
runaway destruction. It is called a reinforcing feedback loop…</p>
<p>Thinking in systems - Donella H. Meadows</p>
</blockquote>
<p class="text-center"><img src="/img/thinking-in-systems-book.jpg" alt="Thinking in Systems's book cover" class="img-fluid mh-25" /></p>
<p>It is a great way to put words close to concepts I am exploring in practice. It
is not a book written for developers, but you know, it works, and you can apply
what you read everywhere.</p>
<p>Words are coming from <a href="https://en.wikipedia.org/wiki/Donella_Meadows">Donatella Meadows</a>, the author of this book, a
scientist, author, teacher, and a system analyst.</p>
<p>If you enjoyed <a href="https://gianarb.it/blog/reactive-planning-and-reconciliation-in-go">“Reactive planning and reconciliation in Go.”</a> this book is a
great way to go deeper into this topic without reading a long book; in the end,
it is less than 200 pages.</p>
<p>Systems are everywhere: the water cycle, the ability our body has to recover,
the evolution. This book helps you understand how to describe and spot them,
giving you the ability to see systems, such as when writing Software or when
looking for alternative solutions.</p>
<p>From my experience, when you can simplify a problem into a system, you get an
entity capable of balancing itself, a more resilient solution, and repeatable
workflow.</p>
<p>Self-balancing as a thermostat, for example. Resilient as a Kubernetes
reconciliation loop, repeatable as the water cycle is or an idempotent server
provisioning.</p>
Maintainer life, be an enablerBe an enabler is important in my daily job. It is a skill I learned as open source maintainer.https://gianarb.it/img/me.jpg2020-08-20T09:08:27+00:002020-08-20T09:08:27+00:00https://gianarb.it/blog/maintainer-life-be-enabler<p>This is not something important only in open source, or as maintainer. But it is
a skill I have personally learned as one.</p>
<p>When I have to build a sustainable open source project, but it applies with
teams as well developing the right code or new feature is often not that useful
in the long term. I think you get quickly to something better when people
collaborate together in an effective way.</p>
<p>The maintainer’s role is to enable other people to contribute successfully.</p>
<p>You have to switch from “let’s write documentation” to “how do I create a
workflow that enables contributors to write documentation.” I started with doc
because I think it is crucial. It is easier to write documentation when you are
writing code or a new feature. And we know a developer prefers to write code; as
a maintainer, you need to create a workflow that allows the contributor to write
documentation quickly when writing code. In practice, mark a PR with a label
<code>needs-doc</code> and make it a requirement for the PR to be merged. The
maintainer has to design a rock-solid structure for the documentation. In this
way, the contributor won’t spend two days trying to figure out where to add the
documentation for its feature.</p>
<p>You can’t ask a contributor to create an entire test suite or to write
documentation if you don’t have one. But from a solid foundation is reasonable
to ask a contributor to keep at least the same quality level.</p>
<p>You don’t write all the tests; you create and maintain the continuous delivery
pipeline required to help contributors to stay compliant. Is your project suffering
from low test coverage? Do not waste time writing all the tests yourself;
codebase is significant, and pull requests are flowing continuously. You have to
stay focused on developing a system that brings and keeps you where you want:
good coverage in this case.</p>
<p>In practice, you can create another label <code>needs-tests</code> to notify the
contributor that its work won’t be merged until tests will be added (the plural
is crucial, tests!). You can use something like <a href="https://codecov.io/">codecov</a>
in your CI to evaluate with numbers the situation. Invest time being sure that
tests are easy to write; write a doc in the contributor file highlighting how to
write a good test. If a package is too hard to test, you can write a few tests
that other people can use as a starting point or a reusable set of utility
functions.</p>
<p>Being a facilitator or an enabler is a lot of work. If you feel less
effective because you wrote 90% of the codebase at this point and you can write
documentation and tests by yourself in a couple of days you are wrong. Or at
least from my experience you can do it but the outcome will be worst in quality
compared with the one you can build from a solid foundation in a collaborative
environment.</p>
Interface segregation in action with GoIt takes a couple of hours to get an hello world up and running in a new language but it takes ages to learn it deeply. Even if Go has a learning curve that is affordable some concepts take time to stick in mind. Interface are everywhere and this flexibility makes them crucial to write maintainable Go code.https://gianarb.it/img/me.jpg2020-08-20T09:08:27+00:002020-08-20T09:08:27+00:00https://gianarb.it/blog/interface-segreation-in-action-with-go<p>Everybody should write an article about Golang interface! I don’t know why I
waited so long for mine!</p>
<p>Golang interfaces are your best friends when it comes to mocking an object or to
specify a well scoped set of functionalities required by a function to interact
with an object.</p>
<p>Yep! That’s how they work, you have an entire object that does a lot of cool
things, but when you pass it to a function only a subset of it get used, that’s
when you can replace the structure itself with an interface that only requires
what it is needed by the structure.</p>
<p>In this way you will have a smaller piece of code to mock in your test and to
deal with (this is a good way to hide functions you don’t want other people or
yourself in a rush to use).</p>
<p>Even more when you remember to keep the interface small via composition.</p>
<p>For example let’s suppose you have to build an interface that describes a generic
resource that you can Create, Update and Delete. This is useful to standardize
something that can be persisted in a database. I am setting this up
so.</p>
<p>You should not use <code>interface{}</code> because it is too generic. I used it for
simplicity but Kubernetes for examples uses an object called
<a href="https://godoc.org/k8s.io/apimachinery/pkg/runtime"><code>runtime.Object</code></a> and it way
better. Go 2 will have generics that will make this situation even easier. Or
you can use code generation as well. But the idea to use a serializable object
like Kubernetes is good.</p>
<pre><code class="language-golang">type Resource interface {
Create(ctx context.Context) error
Update(ctx context.Context, updated interface{}) error
Delete(ctx context.Context) error
}
</code></pre>
<p>This is a reasonably small interface, it is easy to satisfy but I do not like
the name. I think it does not give me the ability to figure out what’s its
purpose. It represents, a resource but I prefer to call interface as actions or
a adjective. In this case the structure who implements this interface can be
stored in a database. I think a better name for it is:
<a href="https://en.wiktionary.org/wiki/persistable">“Persistable”</a> because it makes
clear its purpose.</p>
<p>A strategy to make an interface smaller in this case is to break it in actions:</p>
<pre><code class="language-golang">type Creatable interface {
Create(ctx context.Context) error
}
type Updatable interface {
Update(ctx context.Context, updated interface{}) error
}
type Deletable interface {
Delete(ctx context.Context) error
}
</code></pre>
<p>And you can use composition to create an interface that requires all the three
actions to work if you need it:</p>
<pre><code class="language-golang">type Persistable interface {
Deletable
Updatable
Creatable
}
</code></pre>
<p>This is useful when a function uses more than one of those actions, if you have
an interface that contains also <code>Get</code> or <code>View</code> you can think about a different
split <code>ReadOnly</code> contains <code>Get</code>, <code>View</code> and <code>Modifiable</code> that will require only
the functions <code>Update</code>, <code>Create</code>, <code>Delete</code>.</p>
<p>Imagine you are writing a set of http handlers to expose a CRUD API around your
resources:</p>
<pre><code>Create
Update
Delete
List
GetByID
</code></pre>
<p>Usually it looks like this, you can create an interface for every function, all
your resources will implement the functions and you will be able to write a
single “Create” handle for all the resources:</p>
<pre><code class="language-golang">func CreateHandle(c Creatable) func(w http.ResponseWriter, r *http.Request) {
return http.HandleFunc("/resource", func(w http.ResponseWriter, r *http.Request) {
if err := c.Create(r.Context); if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusCreated)
})
}
</code></pre>
<p>If you have to write a test for the handler it does not matter how complicated
the resource is, you just have to mock the <code>Creatable</code> interface, one single
function. This is a very basic example, if you need to add validation the
<code>Creatable</code> function can require a <code>func Valid() error</code> that you can
add incrementally in all your resources.</p>
<pre><code class="language-golang">func CreateHandle(c Creatable) func(w http.ResponseWriter, r *http.Request) {
return http.HandleFunc("/resource", func(w http.ResponseWriter, r *http.Request) {
if err := c.Valid(); err != nil {
w.WriteHeader(http.StatusBadRequest)
return
}
if err := c.Create(r.Context); if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusCreated)
})
}
</code></pre>
E2E testing Tinkerbell Setup tutorial in GoMy takeaway from having a to write a end to end test for the Tinkerbell Vagrant setup tutorial. How I wrote it and why, lesson learned and tips.https://gianarb.it/img/me.jpg2020-08-03T09:08:27+00:002020-08-03T09:08:27+00:00https://gianarb.it/blog/e2e-test-tinkerbell-vagrant-setup-with-go<p><a href="https://tinkerbell.org">Tinkerbell</a> is a tool open sourced recently by
<a href="https://packet.com">Packet, an Equinix company</a>, the company I work for.</p>
<p>It is a provisioner for bare metal. You can switch servers on and off via API,
executing workflows and install operating systems on a server that does not have
one!</p>
<p>Tinkerbell is in its early days as open source project but the concept is battle
tested from 6 years of production use internally at Packet.</p>
<p>I am excited to learn a lot of the cool technologies that are making datacenters
working, but I am not here to write about it<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>.</p>
<p>One of my recent tasks<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup> was about end to end testing the Vagrant Setup
tutorial<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup> we wrote.</p>
<p>I like the idea! The Setup tutorial is important for our community because it is
the entry point for a lot of people and having a consistent way to test its
accuracy is crucial.</p>
<p>It is also a quick way to get a valuable end to end test running that covers the
entire project, at a high level.</p>
<p>Tinkerbell is under development and it is easy to make mistakes and break
things at this point, we have to know when it happens. Tinkerbell requires
virtualisation capabilities, and we do not have an end to end testing framework
for that yet.</p>
<h2 id="tell-me-more-about-the-test-itself">Tell me more about the test itself</h2>
<p>It is a long tasks but let’s summarize it (have a look at the tutorial, it helps
to read this article moving forward):</p>
<ol>
<li>The script has to start a vagrant machine called provisioner</li>
<li>When the provisioner is up it has to exec via ssh a docker-compose command
that starts a bunch of containers, one of those is Tink grpc server</li>
<li>When Tinkerbell is up and running we have to do a bunch of things like:
a. Register a new hardware
b. Create a template
c. Create the workflow that will get executed in the worker from a template</li>
<li>Start the worker</li>
<li>Wait and check if the workflows executes as expected.</li>
</ol>
<p>NOTE: the test should clean up after itself, Vagrant is not ideal
to get parallelization of VMs, and we do not support it. A dirty environment
will break future tests as it is today.</p>
<h2 id="how-to-write-this-test">How to write this test</h2>
<p>There are a million way to write end to end test the one I evaluated are bash
and Go.</p>
<p>The project is in Go, Tinkerbell serves a gRPC server and a client, I thought it
was a good idea to write everything in Go to try the client itself and because
it is easier to coordinate long running actions with channels and context
compared with bash for example. Or at least that’s what I think.</p>
<p>I can also keep the code inside the <code>testing</code> framework that Go provides keeping
the test closer to the code and the developers that contribute to the project,
compared with a random <code>scripts.sh</code>.</p>
<p>I am not sure if this will be useful in the future but one of my goal was to
serve a clean API and a small framework that can be used to write other tests
that starts from the Vagrant setup. This is the API I designed:</p>
<pre><code class="language-go">type Vagrant struct {}
func Up(ctx context.Context, opts ...VagrantOpt) (*Vagrant, error) {}
func (v *Vagrant) Destroy(ctx context.Context) error {}
func (v *Vagrant) Exec(ctx context.Context, args ...string) ([]byte, error) {}
</code></pre>
<p>Consistency is important, developers who knows vagrant or that will have to fix
the tests coming from the tutorial will know <code>Up</code>, <code>Destroy</code> and <code>Exec</code> because
those verbs are used by Vagrant and in the documentation itself.</p>
<p>Even for Go developers <code>Exec</code> is not a new function, <code>os/exec</code><sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">4</a></sup> exists and it
does a similar job, the one I wrote is over ssh.</p>
<p>This library now has its own repository:
<a href="https://github.com/gianarb/vagrant-go">gianarb/vagrant-go</a>.</p>
<h2 id="go-challenges-and-tips-and-tricks">Go challenges and tips and tricks</h2>
<p>I would like to share some of the challenges I faced when writing the Vagrant
framework and some tips useful for this task.</p>
<h2 id="opt-are-great">Opt are great!</h2>
<p>I have to say options are great! It is a well known pattern in Go and it
translates to:</p>
<pre><code class="language-go">ctx := context.Background()
machine, err := vagrant.Up(ctx,
vagrant.WithLogger(t.Logf),
vagrant.WithMachineName("provisioner"),
vagrant.WithWorkdir("../../deploy/vagrant"),
)
if err != nil {
t.Fatal(err)
}
</code></pre>
<p>It allowed me to add new options and to tune the Vagrant struct with strong
default. If you never used it, do it! It is pretty easy, you need an interface
like this:</p>
<pre><code class="language-go">type VagrantOpt func(*Vagrant)
</code></pre>
<p>In this way you can write as many <code>With</code>function you need:</p>
<pre><code class="language-go">func WithStderr(s io.ReadWriter) VagrantOpt {
return func(v *Vagrant) {
v.Stderr = s
}
}
func RunAsync() VagrantOpt {
return func(v *Vagrant) {
v.async = true
}
}
</code></pre>
<p>I execute the opts as part of the <code>Up</code> function:</p>
<pre><code class="language-go">func Up(ctx context.Context, opts ...VagrantOpt) (*Vagrant, error) {
const (
defaultVagrantBin = "vagrant"
defaultName = "vagrant"
defaultWorkdir = "."
)
v := &Vagrant{
VagrantBinPath: defaultVagrantBin,
Name: defaultName,
Workdir: defaultWorkdir,
log: func(format string, args ...interface{}) {
fmt.Println(fmt.Sprintf(format, args))
},
}
for _, opt := range opts {
opt(v)
}
// ...
}
</code></pre>
<h3 id="test-segmentation-with-packages">test segmentation with packages</h3>
<p>I don’t want to run the vagrant end to end tests as part of the default test
suite because they take too much time and they require Vagrant installed. They
do not even run in CI in the same way unit test works, but I will get to it
later.</p>
<p>I learned that packages that starts with <code>_</code> does not get executed when using
something like <code>./...</code>.</p>
<p>I wrote the framework and tests as part of the package:</p>
<pre><code class="language-console">./test/_vagrant/
./vagrant.go
./vagrant_test.go
</code></pre>
<p>In this way to run the tests you have to explicitly call the package out:</p>
<pre><code class="language-console">$ go test ./test/_vagrant
</code></pre>
<h3 id="observability-or-what-is-going-on">Observability or “what is going on?”</h3>
<p>Go has its own way to print logs during the execution of the tests:</p>
<pre><code class="language-console">$ go test -v ./...
</code></pre>
<p>It works because <code>testing</code> has a function called <code>t.Log</code> and <code>t.Logf</code>. Those
functions watches the <code>-v</code> flags. To be complaint with that and to keep the
<code>Vagrant</code> struct agnostic I wrote a <code>WithLogger</code>:</p>
<pre><code class="language-go">func WithLogger(log func(string, ...interface{})) VagrantOpt {
return func(v *Vagrant) {
v.log = log
}
}
</code></pre>
<p>The function it accepts as a argument is <code>t.Logf</code>.</p>
<p>Continuous Integrations runs with verbosity enabled for this task because it is
long and complicated, the logging prints all the outputs from the <code>vagrant up</code>
and <code>destroy</code> command, and the stdout for the <code>exec</code> over ssh, it gives a very
good overview about what is going on.</p>
<h3 id="stdout-and-stdin-buffer-and-loggers">Stdout and Stdin, buffer and loggers</h3>
<p>I don’t have a lot to say about this other than: “it was very hard to do!!”.
The code that fixed my problems can be summarized in this way:</p>
<pre><code class="language-go">stderrPipe, err := cmd.StderrPipe()
if err != nil {
return nil, fmt.Errorf("exec error: %v", err)
}
stdoutPipe, err := cmd.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("exec error: %v", err)
}
go v.pipeOutput(ctx, fmt.Sprintf("%s stderr", cmd.String()), bufio.NewScanner(stderrPipe))
go v.pipeOutput(ctx, fmt.Sprintf("%s stdout", cmd.String()), bufio.NewScanner(stdoutPipe))
err = cmd.Start()
</code></pre>
<pre><code class="language-go">func (v *Vagrant) pipeOutput(ctx context.Context, name string, scanner *bufio.Scanner) {
for scanner.Scan() {
select {
case <-ctx.Done():
return
default:
v.log("[pipeOutput %s] %s", name, scanner.Text())
}
}
}
</code></pre>
<h3 id="kill-process-and-subprocess">Kill process and subprocess</h3>
<p>There are a lot of process going on when creating or destroying a VM with
Vagrant. There is VirtualBox for example, and we have an edge case for the
worker machine because the <code>up</code> commands technically never ends, it is in
pending until you <code>destroy</code> the machine. But you can’t run multiple commands
against the same machine because <code>up</code> holds a lock and it blocks <code>destroy</code> to
execute. <code>os/exec</code> helps here but you have to tune it a little bit:</p>
<pre><code class="language-go">cmd := exec.CommandContext(ctx, v.VagrantBinPath, args...)
cmd.Dir = v.Workdir
cmd.Stdout = v.Stdout
cmd.Stderr = v.Stderr
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
</code></pre>
<p>Now when killing <code>cmd</code> the subprocess terminates as well.</p>
<h2 id="continuous-integration">Continuous Integration</h2>
<p>We decided to go with GitHub Actions with a self running runner, in this way we
can use Packet bare metal that supports virtualisation.</p>
<p>As I told you I don’t want this test to run for all the commit, or for all the
pull request because it is time and resource consuming. It is also risky, so I
want maintainers to decide when to trigger it.</p>
<p>That’s why it gets triggered with a GitHub label:</p>
<pre><code class="language-yaml">name: Setup with Vagrant on Packet
on:
push:
pull_request:
types: [labeled]
jobs:
vagrant-setup:
if: contains(github.event.pull_request.labels.*.name, 'ci-check/vagrant-setup')
runs-on: self-hosted
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Vagrant Test
run: |
export VAGRANT_DEFAULT_PROVIDER="virtualbox"
go test -v ./test/_vagrant
</code></pre>
<p>This is what it takes to make the process working!! And I am still surprised it
is so easy! When a contributor label a PR with <code>ci-check/vagrant-setup</code> the
process starts. My idea was to remove the label straight away, but I am
<a href="https://github.community/t/actions-ecosystem-action-remove-labels-fails-resource-not-accessible-by-integration/124188">blocked</a>.</p>
<p>An alternative that we are evaluating is to run it as a cronjob<sup id="fnref:5" role="doc-noteref"><a href="#fn:5" class="footnote" rel="footnote">5</a></sup> as well.</p>
<h2 id="testing-is-the-real-power">Testing is the real power</h2>
<p>E2E testing are fun to write because they bring a lot of challenges in terms of
coordination and stability. You have to write good code in order to make them
stable. I hope you learned something from my experience and if you have any
question let me know <a href="https://twitter.com/gianarb">here</a>. I am happy to go
deeper on some of those topics based on your suggestions.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>If you are curious ask me any question on Twitter @gianarb <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>https://github.com/tinkerbell/sandbox/pull/7 <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>https://tinkerbell.org/setup/local-with-vagrant/ <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>https://golang.org/pkg/os/exec/#pkg-examples <a href="#fnref:4" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:5" role="doc-endnote">
<p>https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#onschedule <a href="#fnref:5" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Show Me Your Code with David McKay (rawkode): Terraform what?I would like to layout a GitHub repository using Terraform that helps me to manage GitHub organization via code in a collaborative way. I am not that good with Terraform, even less when it comes to Terraform 0.12 and all the sweet things it does. My friend David McKay aka @rawkode is way better than myself and he will teach me a bunch of things live on Twitchhttps://gianarb.it/img/show-me-your-code-logo.png2020-07-29T09:00:27+00:002020-07-29T09:00:27+00:00https://gianarb.it/blog/terraform-what-with-rawkode<blockquote class="twitter-tweet tw-align-center"><p lang="en" dir="ltr">🆘Tomorrow is
"Show me your code" time! ⌛️Live on Twitch!<br /><br />Who is the guest?
The unique <a href="https://twitter.com/rawkode?ref_src=twsrc%5Etfw">@rawkode</a> !<br />What is
all about? David will teach <a href="https://twitter.com/hashtag/Terraform?src=hash&ref_src=twsrc%5Etfw">#Terraform</a>
to a newbie (me!) and I hope to learn how to manage a GitHub organization as
code! <a href="https://t.co/jNbq7uoAD3">https://t.co/jNbq7uoAD3</a><br />See you
there! 🖥️ <a href="https://t.co/b2P4bj8Tki">pic.twitter.com/b2P4bj8Tki</a></p>— gianarb
(@GianArb) <a href="https://twitter.com/GianArb/status/1288511212351901697?ref_src=twsrc%5Etfw">July
29, 2020</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Let’s suppose I don’t know a lot about Terraform because infrastructure as code
is not something that makes me happy. I know it is useful. There is a lot of
cool things in Terraform itself and I should know it more.</p>
<p>I happen to know and work with David that is always on top of the topic! I had
an idea, and he will help me to validate it live tomorrow August 29th in the
afternoon Central Europe.</p>
<p>We will put together a repository of “GitHub as Code” that we can use to
manage collaboratively a GitHub organisations:</p>
<ol>
<li>Adding new members</li>
<li>Creating new teams</li>
<li>New repositories</li>
<li>Importing already created resources from GitHub to Terraform</li>
<li>If we can have a look at CI/CD with GitHub action and/or Terraform Cloud.</li>
</ol>
<p>I will try first time <a href="https://visualstudio.microsoft.com/services/live-share/">Visual Studio Code with Live
Share</a> and I think
David will teach me a lot of things about Terraform and 0.12 syntax.</p>
<p>This is something I hope we can use with
<a href="https://github.com/tinkerbell">Tinkerbell</a>.</p>
<h2 id="david-mckey-rawkode">David McKey (rawkode)</h2>
<p>David McKay is a technologist from Glasgow, Scotland. Currently, working at
Packet as Senior Tech Evangelist. Well known on Twitter as
<a href="https://twitter.com/rawkode">@rawkode</a> and he writes on
<a href="https://rawkode.com/articles">rawkode.com</a>.</p>
<h2 id="links">Links</h2>
<ul>
<li>We will start from a nosense prototype I did yesterday
<a href="https://github.com/gianarb/terraformy-github-org">github.com/gianarb/terraformy-github-org</a></li>
<li>We will use Terraform 0.12 with the <a href="https://www.terraform.io/docs/providers/github/index.html">GitHub provider</a></li>
<li>Have a look at what we are doing at Packet with
<a href="https://tinkerbell.org">Tinkerbell</a></li>
</ul>
First journeys with netboot and ipxe installing UbuntuI have started to experiment with netbot, ipxe and OS automation recently to better understand how bare metal provisioning works. I got to a point where I am able to install Ubuntu automatically via iPXE and preseed. This is an article about how and a bit of why.https://gianarb.it/img/me.jpg2020-06-10T09:08:27+00:002020-06-10T09:08:27+00:00https://gianarb.it/blog/first-journeys-with-netboot-ipxe<p>I recently joined Packet, a company acquired by Equinix, finally after 7 years
working with cloud computing I can see how they look like from the other side!</p>
<blockquote class="blockquote text-center text-center text-center text-center text-center text-center text-center">
<p>Spoiler alert: clouds are made of servers</p>
</blockquote>
<p>As a first task I had to revamp the kubernetes cluster-api implementation,
moving it from v1alpha1 to v1alpha3. Kind of cool and in a domain I know very
well. We tagged the first release and I can’t wait to see how it will be used.
I got some meetups and webinars planned about it, so stay in touch on
<a href="https://twitter.com/gianarb">twitter</a> to know more about it.</p>
<p>Anyway, one of the topics I am curious about is hardware automation. The idea to
get up and running, in a repeatable and autonomous way a piece of inanimate
metal well known as rack, switch, server is a topic I never touched and I would
like to know more! Obviously, this is only one of the articles I will write about
the topic. Mainly because there is much to learn.</p>
<p>As you can image when we buy a server it does not do much, it’s great to keep
your door open, and as a table. It has to be configured, in our case customers
can do it via API, it means there is some code involved! I want to know more!!</p>
<p>For sure there are a couple of things that you have to do manually, assemble the sever,
power it on, plug the ethernet cable in, pick the right location and things like
that.</p>
<p>But as you can imagine it comes without operating system, even more complicated to
install because the customers can select the one they like most, or even push
their own one. This is for sure something that has to be done magically, I doubt
we have people running with USB stick in a datacenter installing operating
systems.</p>
<p class="blockquote text-center text-center text-center text-center text-center text-center text-center"><img src="https://i0.wp.com/www.anonimacinefili.it/wp-content/uploads/2019/07/forrest-gump-25-anni.jpg?fit=1200%2C600" alt="Forst Gump picture" class="img-fluid" /></p>
<p>One of the things that happen when booting a laptop or server
is the bootloader. The one that requires a master skill to get in because there
is a timeout, and I never know what to press! I have to be honest I thought they
were pretty static and not that fun.</p>
<p>BUT, there are smart boot-loaders! We know what smart means does days: internet! In
practice there is a bootloader capable of booting not from USB, not from disk
but from the internet.</p>
<p>Usually a private network but it is not mandatory, and we spend the last couple
of years doing <code>curl something.com | bash</code>. The bootloader downloads a
<code>kernet</code>, the <code>initrd</code> and it will <code>boot</code> an operating system. It is like having a
USB stick that starts the live installation of Ubuntu, there is a lot more after
that because we have to persist the installation on disk, format them and so on.</p>
<p>In this article I will show you at first how to get to something that looks like
like the installation wizard for ubuntu, and also how to automate the
installation via preseed.</p>
<p>The bootloader is called PXE, and the new generation I used is called iPXE.</p>
<p>The cool things about PXE/iPXE is that you can chain scripts from one to
another. <a href="https://twitter.com/grhmc">Graham Christensen</a> told me that the main
goal when you work with PXE is to escape from it and get to iPXE, that is way
cooler. Time for a recap:</p>
<ol>
<li>The machine starts and enters PXE</li>
<li>PXE download and chains iPXE in this way we are on iPXE bootload</li>
<li>From iPXE you can download kernel, initrd and boot the OS in RAM.</li>
</ol>
<p>IPXE supports different ways to download what it needs from the internet, the
one I used so far are TFTP and HTTP.</p>
<h2 id="what-is-this-pxeipxe">What is this PXE/iPXE?</h2>
<p>Such a nice question, I had the same one few days ago. A couple of links:</p>
<ol>
<li><a href="https://en.wikipedia.org/wiki/Preboot_Execution_Environment">Wikipedia: Preboot Execution Environment</a></li>
<li><a href="https://ipxe.org/">iPXE: open soure boot firmware</a></li>
</ol>
<p>Roughly you can think about iPXE as a shell that has a bunch of commands like:</p>
<ol>
<li>dhcp: to require an IP from a DHCP server and configures the network
interface</li>
<li>route: to figure out if the network interface is configured (if it has an IP
already)</li>
<li>chain: gets an argument (a URL) and it executes the content, it is a good way
to pass scripts</li>
<li>You can see variables <code>set name value</code></li>
<li>kernet: downloads the kernel from a source and load it</li>
<li>initrd: download the init ramdisk</li>
<li>boot: triggers the boot</li>
</ol>
<p>And <a href="https://ipxe.org/cmd">many more</a> that I did not use yet but docs lists
them.</p>
<p>It also has support for building a menu like this one:</p>
<p class="blockquote text-center text-center text-center text-center text-center text-center text-center"><img src="https://netboot.xyz/images/netboot.xyz.gif" alt="netboot menu" class="img-fluid" /></p>
<p>The image comes from <a href="https://netboot.xyz/">netboot.xyz</a> as you can see from
their website is a project that simplify the process of installing a lot of
different operating systems via PXE. I started with it at first for my
experiments. Obviously menus and automations does not play nice together but in
the process of learning I took this extra step.</p>
<h2 id="hello-world">Hello world</h2>
<p>To give you some context this is the script I used to start the installation
wizard for Ubuntu:</p>
<pre><code>#!ipxe
dhcp net0
set base-url http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/legacy-images/netboot/ubuntu-installer/amd64/
kernel ${base-url}/linux console=ttyS1,115200n8
initrd ${base-url}/initrd.gz
boot
</code></pre>
<p>I hope it looks familiar, some code at least. In order to
reach the internet you need an IP, and to get one if you are lazy like me is to
use a DHCP (the alternative is to set one statically). The first command does that, asks the DHCP an IP for the network
interface <code>net0</code>.</p>
<p>When the IP is set, iPXE reachs <code>ubuntu.com</code> to get the kernel and the initrd.
Everything I need to boot an OS in RAM.</p>
<h2 id="lets-try">let’s try</h2>
<p>I am using <a href="https://packet.com">Packet</a> for my tests because it serves the low
level capabilities I need, it supports the server creation without OS and with
iPXE. You can register and do it yourself, <code>gophernetes</code> is a coupon that will
give you 30$ credit.</p>
<p>When you request a device (a server) on Packet you can select the operating
system, we don’t need to do it, so you can select <code>Custom iPXE</code> because we are
going to install it ourselves.</p>
<p class="blockquote text-center text-center text-center text-center text-center text-center text-center"><img src="/img/packet-create-device.png" alt="A screenshot from packet.com about how to create an on demand device with Custom iPXE" class="img-fluid w-75" /></p>
<p>There are two ways we can inject out script to iPXE in order to teach the server
what to do when it boot, first one is giving a URL (I use a gist (raw link)), or
via user data. The script I used is the one pasted above. You can create a gist
and paste the link in “iPXE Script URL” or you can use the user data, as I am
doing right now.</p>
<p class="blockquote text-center text-center text-center text-center text-center text-center text-center"><img src="/img/packet-user-data.png" alt="A screenshot form packet.com about how to pass a user data to a server" class="img-fluid w-75" /></p>
<p>As soon as the machine starts you can click on its name to enter the get its
details and you get a ssh into the “Out-of-Band Console” console:</p>
<p class="blockquote text-center text-center text-center text-center text-center text-center text-center"><img src="/img/packet-out-of-band.png" alt="A screenshot from packet.com that shows where to locate the out-of-band
console" class="img-fluid w-75" /></p>
<p class="blockquote text-center text-center text-center text-center text-center text-center text-center"><img src="/img/packet-out-of-band-ssh.png" alt="A screenshot from packet.com that shows how to get the ssh command to use
the out-of-bound console" class="img-fluid w-75" /></p>
<p>ALERT: if you are doing this activity, remember to enable OpenSSH when you
follow the installation wizard otherwise you won’t be able to SSH in the server
at the end!</p>
<p>When deploying the server the code you passed gets chained from the Packet iPXE.
And you should see the Ubuntu wizard ready for you:</p>
<p class="blockquote text-center text-center text-center text-center text-center text-center text-center"><img src="/img/packet-ubuntu-install-wizard.png" alt="A screenshot from my terminal that shows the first wizard for ubuntu" class="img-fluid" /></p>
<p>At the end of the wizard you will get a persisted operating system in the server
itself, it will survive the reboot and it will be just as any other server you
used in the past, but better because you know how you installed the OS! Get its
IP and ssh in!</p>
<h2 id="preseed">Preseed</h2>
<p>Debin like operating system, like Ubuntu support a technology called
<a href="https://help.ubuntu.com/lts/installation-guide/s390x/apb.html">preseed</a>, it
practice is a text file that contains the answers for all the questions the
Ubuntu wizard makes.</p>
<p>In this way no point and click is required. I put together a file here and I
uploaded it as a gist:</p>
<pre><code>#### Contents of the preconfiguration file (for stretch)
### Localization
# Preseeding only locale sets language, country and locale.
d-i debian-installer/locale string en_US.UTF-8
d-i localechooser/supported-locales multiselect en_US.UTF-8
d-i console-setup/ask_detect boolean false
d-i keyboard-configuration/xkb-keymap select GB
# Keyboard selection.
# Disable automatic (interactive) keymap detection.
d-i console-setup/ask_detect boolean false
d-i keyboard-configuration/xkb-keymap select us
# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
d-i netcfg/choose_interface select auto
# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
# Disable that annoying WEP key dialog.
d-i netcfg/wireless_wep string
### Mirror settings
d-i mirror/country string manual
d-i mirror/http/hostname string archive.ubuntu.com
d-i mirror/http/directory string /ubuntu
d-i mirror/http/proxy string
# Root password, either in clear text
d-i passwd/root-password password rootroot
#d-i passwd/root-password-again password rootroot
# or encrypted using a crypt(3) hash.
#d-i passwd/root-password-crypted password [crypt(3) hash]
# To create a normal user account.
d-i passwd/user-fullname string yay
d-i passwd/username string yay
# Normal user's password, either in clear text
d-i passwd/user-password password norootnoroot
d-i passwd/user-password-again password norootnoroot
# Set to true if you want to encrypt the first user's home directory.
d-i user-setup/encrypt-home boolean false
### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true
# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string US/Eastern
# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean true
# LG provided NTP, should be replaced!
d-i clock-setup/ntp-server string ntp.ubuntu.com
### Partitioning
d-i preseed/early_command string umount /media || true
d-i partman-auto/method string lvm
d-i partman-auto-lvm/guided_size string max
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
d-i partman-auto-lvm/new_vg_name string main
d-i partman-md/device_remove_md boolean true
d-i partman-md/confirm boolean true
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
d-i partman-basicmethods/method_only boolean false
### Partitioning
d-i partman-auto/method string lvm
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
### Package selection
tasksel tasksel/first multiselect ubuntu-desktop
# Individual additional packages to install
d-i pkgsel/include string openssh-server build-essential
# Whether to upgrade packages after debootstrap.
# Allowed values: none, safe-upgrade, full-upgrade
d-i pkgsel/upgrade select full-upgrade
d-i pkgsel/update-policy select none
# Individual additional packages to install
d-i pkgsel/include string openssh-server \
vim \
git \
tmux \
build-essential \
telnet \
wget \
curl
# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true
# This one makes grub-installer install to the MBR if it also finds some other
# OS, which is less safe as it might not be able to boot that other OS.
d-i grub-installer/with_other_os boolean true
# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note
</code></pre>
<p>It is a bit weird but if you are familiar with
the Ubuntu installation process I am sure you can spot some similarity.</p>
<p>At this point we have to pass some <code>cmdline</code> arguments to the kernel in order to
have it downloading the preseed file from a raw gist and to tell the kernel that
the installation is automatic:</p>
<pre><code>#!ipxe
dhcp net0
set base-url http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/legacy-images/netboot/ubuntu-installer/amd64/
set preseed-url https://gist.githubusercontent.com/gianarb/acea1ca5b73a318fd74cbb002cae21f3/raw/76e5d036ee28c485cc7cf42a317c99e678f08a6c/ubuntu.preseed
kernel ${base-url}/linux console=ttyS1,115200n8 auto=true fb=false priority=critical preseed/locale=en_GB url=${preseed-url} DEBCONF_DEBUG=5
initrd ${base-url}/initrd.gz
boot
</code></pre>
<p>The mechanism is the same as before, you can create a gist and link it during
the server creation or you can paste this as cloud init.</p>
<p>At this point you can connect to the <code>Out of Band</code> console via ssh and the
installation wizard will look like a movie! When the process is over the server
reboots and you will be able to SSH in using username: <code>yay</code>, password
<code>norootnoroot</code>. If you are looking for the root password have a look at the
preseed file, the answer is there!</p>
<p>Preseed is probably not what you want at the end, but it is an easy enough way
to get to a persisted OS. It does a lot runtime, by consequence it is time
consuming and it can be flaky when reaching the network.
<a href="https://twitter.com/thebsdbox">Dan</a> pointed me to other ways to do it using
<code>raw</code> images but probably I will experiment moving forward.</p>
<h2 id="conclusion">Conclusion</h2>
<p>That’s it, this is a layer I am not familiar with Packet has an open
source project called <a href="https://tinkerbell.org/">Tinkerbell</a>, that does bare metal
provisioning.</p>
<p>I want to know what it does under the hood! In practice is an open source
version of the provisioner used internally to set up servers. We are moving
towards to it as well!</p>
<p>A lot of the underline technologies like preseed, PXE are 20 years old, and as I
like to say: “I have a lot of new things to learn from the 80s”.</p>
<p>Don’t know where this will bring me but I think the next articles will look
like:</p>
<ol>
<li>How to get a iPXE server to serve my own kernel, initrd</li>
<li>How to get a set of RPIs provisioned</li>
</ol>
<p>Point me to the right direction or if you are curious to know more about this
topic.</p>
Show Me Your Code with Ivan Pedrazas: Application Lifecycle and GitOpsWe are gonna have a chat with Ivan Pedrazas, we will we can go over a simple API how to build, how to deploy and the issues around it and how GitOps helps.https://gianarb.it/img/show-me-your-code-logo.png2020-05-05T09:00:27+00:002020-05-05T09:00:27+00:00https://gianarb.it/blog/show-me-your-code-application-lifecycle-gitops-with-ivan<p>Everyone speaks about GitOps those days! Everyone in their own way. Some
people think that GitOps means pulling git from inside a kubernetes cluster,
other people think it has to be done in CI.</p>
<p>I don’t know how it works or how it should work but my friend Ivan Pedrazas
(@ipedrazas) knows, so we are gonna learn from it.</p>
<p>The idea is to take a simple application and try to figure out how to apply
GitOps to it. Let’s see how far we go.</p>
<blockquote class="twitter-tweet tw-align-center"><p lang="en" dir="ltr">My plan is to show a
simple app (frontend in vue.js and a flask API), how to build the different
components, how to deploy them using GitOps. Finally, we will modify different
parts of the app and build and deploy and rinse and repeat :)</p>— Ivan
Pedrazas (@ipedrazas) <a href="https://twitter.com/ipedrazas/status/1266013907057065985?ref_src=twsrc%5Etfw">May
28, 2020</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<h2 id="links">Links</h2>
<ul>
<li><a href="https://ismenta.slack.com/join/shared_invite/zt-ex04og6p-i9GsneKysUCHc3g6su7y1Q#/">Menta - Slack Channel</a></li>
<li><a href="https://gitops-community.github.io/kit/#gitops-days-2020-youtube-playlist">https://gitops-community.github.io/kit/#gitops-days-2020-youtube-playlist</a></li>
</ul>
Show Me Your Code with Enrique Paredes: Kubernetes Permission ManagerEnrique will share tips and code around kubernetes permission manager a project that brings sanity to Kubernetes RBAC and Users management, Web UI FTWhttps://gianarb.it/img/show-me-your-code-logo.png2020-05-05T09:00:27+00:002020-05-05T09:00:27+00:00https://gianarb.it/blog/show-me-your-code-kubernetes-permission-manager<p>Kubernetes when it comes to authentication and authorization is extremely
complicated.</p>
<p>I think its philosophy is well described in the documentation:</p>
<blockquote>
<p>Normal users are assumed to be managed by an outside, independent service. An
admin distributing private keys, a user store like Keystone or Google
Accounts, even a file with a list of usernames and passwords. In this regard,
Kubernetes does not have objects which represent normal user accounts. Normal
users cannot be added to a cluster through an API call.</p>
</blockquote>
<p>Kubernetes has user but they have to come from the outside, it is not its
business to care about them. For authorization it uses RBAC and you have a very
long list of possibilities and combinations between actions like: LIST, WATCH,
CREATE, DELETE and resources: pods, deployments, ingress, services…</p>
<p>Sighup is a company based in Italy and well known for their contributions to the
Cloud Native ecosystem. One of their last project is called
<a href="https://github.com/sighupio/permission-manager">permission-manager</a>, it is open
source and it can be describe as follow: “it is a project that brings sanity to
Kubernetes RBAC and Users management, Web UI FTW”.</p>
<p>I will host Enrique (<a href="https://twitter.com/iknite">@twitter</a>) to talk about the
challenges he had when writing such a crucial project, hoping to see some code!</p>
<p>Links:</p>
<ul>
<li><a href="https://kubernetes.io/docs/reference/access-authn-authz/authentication/">Kubernetes
Authentication</a></li>
<li><a href="https://sighup.io/">Sighup webiste</a></li>
<li><a href="https://github.com/sighupio/permission-manager">github.com/sighupio/permission-manager</a></li>
</ul>
Show Me Your Code with Philippe and Giacomo: Vault plugin for WireguardShow me your code has two special guest Giacomo Tirabassi from InfluxData and Scorsolini from Sighup. We will code a new Hashicorp Vault plugin to generate Wireguard configuration. Vault is a popular secret storage developed in open source. Wireguard is networking module and VPN part of the Linux kernel. Let's have some funhttps://gianarb.it/img/show-me-your-code-logo.png2020-04-30T09:00:27+00:002020-04-30T09:00:27+00:00https://gianarb.it/blog/show-me-your-code-vault-wireguard<p>When: Friday 1st May 10am GMT+2 (4am EDT)</p>
<h2 id="lets-write-a-vault-plugin-for-wireguard">Let’s write a Vault plugin for Wireguard</h2>
<p>1st of May. We are on vacation, the perfect day to start a side project.</p>
<p>Giacomo and Philippe started coding an integration between <a href="https://www.vaultproject.io/">HashiCorp
Vault</a> and
<a href="https://www.wireguard.com/">Wireguard</a>. This is great by itself. Vault is cool,
Wireguard is awesome, what do you need more?</p>
<p>The integration is Vault plugin, and we decided to stream the session on
<a href="https://twitch.tv/gianarb">Twitch</a> because that’s what cool kids do those days.</p>
<p>We actually made it, as every side project it is not ready at all, but we got
the boilerplate code required by a Vault Plugin to work and the project is
available on GitHub
<a href="https://github.com/gitirabassi/vault-plugin-secrets-wireguard">gitirabassi/vault-plugin-secrets-wireguard)</a>.</p>
<p>After more than an hour of fun we had to stop but they promise we will have a
follow up meeting as soon as they have an E2E workflow to show me!</p>
<h2 id="about-giacomo">About Giacomo</h2>
<p><a href="https://twitter.com/gitirabassi">Giacomo</a> works as Site Reliability Engineer at
InfluxData. He is an expert on Kubernetes (💼 AWS DA, CKA, CKAD), containers,
Terraform and everything coming from Hashicorp! Traveler and cooker, in a
quest for flavors 🤖</p>
<h2 id="about-philippe">About Philippe</h2>
<p><a href="https://twitter.com/Phisc0">Philippe</a> works as DevOps engineer at Sighup.
Computer Science and Engineering M.Sc. Student @ Politecnico di Milano. Linux
user, open source lover and new technologies’ explorer.</p>
<h2 id="links">Links</h2>
<ul>
<li><a href="https://github.com/gitirabassi/vault-plugin-secrets-wireguard">GitHub repository for the project</a></li>
<li><a href="https://www.vaultproject.io/">HashiCorp Vault</a></li>
<li><a href="https://www.wireguard.com/">Wireguard</a></li>
<li><a href="https://www.vaultproject.io/docs/internals/plugins">Vault: Plugin System</a></li>
</ul>
How to write documentation efficientlyA bunch of experiences and considerations about how to write documentation efficiently. Without wasting too much time or even more important, without getting too bored or stressed out.https://gianarb.it/img/me.jpg2020-04-18T09:00:27+00:002020-04-18T09:00:27+00:00https://gianarb.it/blog/how-to-write-documentation-efficiently<p>You have to remember two things to effectively read this article:</p>
<ol>
<li>I have a blog post where I create content with a good frequency, and I do it
for fun, so I like to write.</li>
<li>I work in remote, the HQ for InfluxData is in San Francisco, it means I am +9
from a lot of my colleagues. Writing is a solid communication channel I use
every day at work because I think it is great, and because I do not have
other alternatives.</li>
</ol>
<p class="text-center text-center"><img src="/img/child-writes.jpg" alt="Child writing on paper" class="img-fluid" /></p>
<h2 id="develop-a-workflow">Develop a workflow</h2>
<p>If you do not like to clean your apartment a strategy you have is to try to keep
it as clean as possible, and in order day by day, in this way you won’t have to
spend a full weekend cleaning every corner of it. Spread a boring task in
a way that won’t make you too tired.
An effective way is to write along the way, side by side with the code you are
developing.</p>
<p>I can highlight a few steps in the process of writing code: analysis, design,
validation, PoC, rollout. Those phases are not unique, they go continuously over
many iteration. I write during all of those steps, many times. Iterations do not
help only your code, they make documentation solid, you can check for typos an
so on.</p>
<p>If you make writing an ongoing process you will find yourself at the end where
the only thing left is to organize and move what you wrote a way that readers
will find familiar.</p>
<h2 id="find-the-right-place">Find the right place</h2>
<p>There are many time of documentation, because there are a lot of stakeholders
and many phases to document (some of them where listed previously).</p>
<p>If I have to think about my stakeholders they are:</p>
<ol>
<li>project managers</li>
<li>documentation team if you are lucky, otherwise let’s say customers or end users.</li>
<li>VP or tech leads.</li>
<li>your teammate or reviewers</li>
</ol>
<p>All those people will enjoy reading a specific point of view, or phase of work.</p>
<p>I think teammate or reviewers are kind of happy to read the process you followed
to design and implement what you wrote, and they will really appreciate to read
inline documentation for your code, doc blocks and so on.</p>
<p>Project Managers will enjoy reading considerations on issues and things like
that, they are super valuable and I end up copy pasting a lot from those
discussions.</p>
<p>End user obviously need a function documentation they they can follow and also a
bit about internal design, monitoring mainly to get them onboard with the work
you developed. It really depends on your audience. We are lucky and we have a
team that is capable of reading code and figure out what we did, but it is a
nice exercise to help them explaining in a good way your work.</p>
<p>VP and tech leaders are usually focused on the design, why you did something in
a way other than another, the trade off you accepted, the one you avoided, why
and how. I like the idea to write this kind of documentation in the code itself.</p>
<p>I am fascinated when I open C codebases where the first thousen lines of code
are documentation. In Go packages can have a file called <code>./doc.go</code> that <code>godoc</code>
will render as a package introduction. If you work with the kind of tech lead or
VP that are not used to read code anymore, you can always copy paste it to
google doc.</p>
<h2 id="write-a-lot">Write a lot</h2>
<p>This point self explains itself. More you write during all the phases if your
work, less you will have to do all together at the end of the code iteration.</p>
<p>Where I usually end up tired about the code I wrote, even more when it takes
weeks, and it is not easy to work on.</p>
<h2 id="pair-on-documentation">Pair on documentation</h2>
<p>I am not a fan of pair programming but recently I changed my mind a little bit,
probably caused by all this social isolation. Before jumping straight on writing
code with my teammate two solid hours over two iterations writing the <code>./doc.go</code>
file together. The outcome made me happy, I hope it will work the same for you.</p>
<p class="text-center text-center"><img src="/img/toomany-files.jpg" alt="Child writing on paper" class="img-fluid" /></p>
<h2 id="conclusion">Conclusion</h2>
<p>This is my experience when writing documentation, but as I said, I love to do
it! Do you have anything to share about it? I am particularly curious about how
and if you READ somebody else documentation, internally written by your teammate
how do you evaluate it and if you have any suggestion to make it more friendly.
Because it is good to write but people has to be able to read it and get what
they need out of it without wasting too much time.</p>
Show Me Your Code with Dan and Walter: How to contribute to OpenTelemetry JSShow me your code has two special guest Walter CTO for CorelyCloud SRL the company behind the CloudConf in Turin and Dan Engineer at Dynatrace Maintainer of OpenTelemetry JS. This during show we will talk about OpenTelemetry and NodeJS. Walter wrote a plugin for instrumenting mongoose with opentelemetry. We are gonna see how he did it, considerations from Dan and so onhttps://gianarb.it/img/show-me-your-code-logo.png2020-04-11T09:00:27+00:002020-04-11T09:00:27+00:00https://gianarb.it/blog/show-me-your-code-otel-nodejs<p>When: Thursday 16th 6-7pm GMT+2 (9am PDT)</p>
<h2 id="opentelemetry-for-js-and-how-to-contribute">OpenTelemetry for JS and how to contribute</h2>
<p>OpenTelemetry is a specification and set of instrumentation libraries developed
in open source from multiple companies such as Google, HoneyComb.io, Dynatrace,
LightStep and many more!</p>
<p>OpenTracing and OpenCensus joined the force, and they started a common project
called OpenTelemetry that I hope will become the way to go in terms of code
instrumentation because I really think it is something we need.</p>
<p>Walter and his team develop in Javascript, frontend and backend and back in the
day we experimented OpenTracing but we had some issue and it was not easy to
pick up at that time. When I tried OpenTelemetry I realized that it was for him.</p>
<p>He tried it out and he wrote its first opentelemetry instrumentation plugin
mongoose, a popular library he uses and that it was not instrumented yet.</p>
<p>Dan will help us to figure how they designed the opentelemetry-js implementation
as it is today, the good the bad and the ugly about this experience. I hope to
get some feedback about roadmap and future development as well now that the
library reached its first release beta.</p>
<h2 id="about-dan">About Dan</h2>
<p>When I was working on my observability workshop Dan gave me a huge help,
drastically increasing my very low experience with NodeJS. Thank you for that.</p>
<p>Dan works as Engineer at Dynatrace, and he maintains the OpenTelemetry JS
library. You can find him on twitter as <a href="https://twitter.com/dyladan">@dyladan</a>
and in <a href="https://gitter.im/open-telemetry/opentelemetry-node">Gitter</a> discussing
opentelemetry.</p>
<h2 id="about-walter">About Walter</h2>
<p>Walter Dal Mut works as a Solutions Architect <a href="https://corley.it/">@Corley SRL</a>.
He is an electronic engineer who moved to Software Engineering and Cloud
Computing Infrastructures. Passionate about technology in general and open
source movement lover.</p>
<p>You can follow him on <a href="https://twitter.com/walterdalmut">Twitter</a>
and <a href="https://github.com/wdalmut">GitHub</a>.</p>
<h2 id="links">Links</h2>
<ul>
<li><a href="https://opentelemetry.io/">opentelemetry.io</a></li>
<li><a href="https://www.dynatrace.com">dynatrace.com</a></li>
<li><a href="https://gianarb.it/blog/how-to-start-with-opentelemetry-in-nodejs">How to start tracing with OpenTelemetry in NodeJS?</a></li>
<li><a href="https://github.com/open-telemetry/opentelemetry-js">github.com/open-telemetry/opentelemetry-js</a></li>
<li><a href="https://github.com/wdalmut/opentelemetry-plugin-mongoose">wdalmut/opentelemetry-plugin-mongoose</a></li>
</ul>
Show Me Your Code with Carlos and Tibor: Chat about GoReleaser and multiarch supportInformal chat with Carlos maintainer for GoReleaser, myself and Tibor from Docker about docker build image with buildx and buildkit to add support for multiple architecures image in GoReleaser.https://gianarb.it/img/show-me-your-code-logo.png2020-04-08T09:00:27+00:002020-04-08T09:00:27+00:00https://gianarb.it/blog/show-me-your-code-goreleaser-buildkit-multi-arch<p>When: Thursday 23th 6-7pm GMT+2 (9am PDT)</p>
<h2 id="goreleaser-and-buildkit">GoReleaser and BuildKit</h2>
<p>The main reason about why I started “Show me your code” is to chat with a couple
of friends from the open source space about what they are doing, what I am doing.
And ideally have a drink: beers, water or coffe based on timezone discussing the same topic.</p>
<p>Me, Carlos and Tibor will meet on Skype for an informal chat about two open
source project I love: GoReleaser and BuildKit. The conversation will be
streamed on Twitch.</p>
<p>You can follow the event live, or the recording will be available here! Watching
it lives will give you the unique opportunity to share your love for those
projects and your feedback about how to support multi arch docker build in
GoReleaser.</p>
<p><strong>IMPORTANT:</strong> The outcome of this conversation will not force in any way Carlos
to do anything! I hope the experience you as attendees and user of GoReleaser
can share and the experience Tibor has with buildkit will drive to a possible
integration. Because I will love to have multi arch support for my releases!</p>
<p>I had this idea because there is a long-standing PR about this
<a href="https://github.com/goreleaser/goreleaser/issues/530">“Support multi-platform docker
images#530”</a> and I am sure
that a discussion all together will be constructive and nice!</p>
<h2 id="about-carlos-and-goreleaser">About Carlos and GoReleaser</h2>
<p>I am very excited to have <a href="https://twitter.com/caarlos0">Carlos</a> with me, I
relay so much on <a href="https://goreleaser.com/">GoReleaser</a> and its integration with
GitHub action to make my development life cycle reliable, repeatable and fast,
and I am happy to have a chat about its project and what he will do next!</p>
<h2 id="about-tibor-from-docker-and-buildkit">About Tibor from Docker and BuildKit</h2>
<p><a href="https://twitter.com/tiborvass">Tibor @tiborvass</a> is a well-known contributor
and maintainer for Docker since the early days. Active on various open source
communities he is now involved with BuildKit as maintainer.</p>
<p>We know each other virtually and thanks to DockerCon and other events since I
joined the Docker Captain program, I am happy to have him around showing
BuildKit, buildx and the multi arch feature.</p>
<h2 id="links">Links</h2>
<ul>
<li><a href="https://github.com/goreleaser/goreleaser">github.com/goreleaser/goreleaser</a></li>
<li><a href="https://github.com/moby/buildkit">github.com/moby/buildkit</a></li>
<li><a href="https://www.youtube.com/watch?v=5KgaisTEzC8">BuildKit: A Modern Builder Toolkit on Top of containerd, Tonis Tiigi & Akihiro Suda</a></li>
<li><a href="https://www.infoq.com/br/presentations/goreleaser-lessons-learned-so-far/">(INFOQ) GoReleaser: lessons learned so far</a></li>
</ul>
How to start tracing with OpenTelemetry in NodeJS?I developed an eight hours workshop about application monitoring and code instrumentation two years ago. This year I updated it to use OpenTelemetry and that's what I learned to instrument a NodeJS application.https://gianarb.it/img/logo/otel-black-stacked.svg2020-04-07T09:08:27+00:002020-04-07T09:08:27+00:00https://gianarb.it/blog/how-to-start-with-opentelemetry-in-nodejs<p>This post is to celebrate the first beta release for the OpenTelemetry NodeJS
application <i class="fas fa-glass-cheers"></i></p>
<p>Recently I developed a workshop about code instrumentation and application
monitoring. It is an 8 hours full immersion on logs, metrics, tracing and so on.
I developed it last year and I gave it twice. Let me know if you are looking for
something like that.</p>
<p>Almost all of it is opensource but I didn’t figure out a good way to make it
usable without my brain for now. This year I updated it to use OpenTelemetry and
InfluxDB v2.</p>
<p>Anyway the application is called
<a href="https://github.com/gianarb/shopmany">ShopMany</a>. This application does
not return any useful information about its state. It is an e-commerce made of a
bunch of services in various languages. Obviously one of them is in NodeJS and
that’s the one I am gonna show you today.</p>
<p><strong>Discaimer</strong>: I can not define myself as a NodeJS developer. I wrote a bunch of
AngularJS single page application back in the day, I wrote some Cordova mobile
applications ages ago. I am not writing any JS production code since 2015 more
or less.</p>
<h2 id="first-approach">First approach</h2>
<p>I concluded the application instrumentation it was the day the maintainers
tagged the first beta release. Overnight I had to update libraries and test
code. Very luckily.</p>
<p>The way I learned about how to properly instrument
<a href="https://github.com/gianarb/shopmany/tree/master/discount">discount</a> required a
lot of digging in the actual
<a href="https://github.com/open-telemetry/opentelemetry-js">opentelemery-js</a> but
luckily for us it has a lot of examples and the library is designed to load a
bunch of useful modules that are able to instrument the application by itself.
The community is very helpful and you can chat via
<a href="https://gitter.im/open-telemetry/opentelemetry-js">Gitter</a>.</p>
<h2 id="getting-started">Getting Started</h2>
<p>I am using ExpressJS and OpenTelemetry has a plugin for it that you can load,
and it instruments the app by itself, same for MongoDB that is the packaged I am
using.</p>
<p>Those are the dependencies I installed in my applications, all of them are
provided by the repository I linked above:</p>
<pre><code>"@opentelemetry/api": "^0.5.0",
"@opentelemetry/exporter-jaeger": "^0.5.0",
"@opentelemetry/node": "^0.5.0",
"@opentelemetry/plugin-http": "^0.5.0",
"@opentelemetry/plugin-mongodb": "^0.5.0",
"@opentelemetry/tracing": "^0.5.0",
"@opentelemetry/plugin-express": "^0.5.0",
</code></pre>
<p>I created a <code>./tracer.js</code> file that initialize the tracer, I have added inline
documentation to explain the crucial part of it:</p>
<pre><code class="language-js">'use strict';
const opentelemetry = require('@opentelemetry/api');
const { NodeTracerProvider } = require('@opentelemetry/node');
const { SimpleSpanProcessor } = require('@opentelemetry/tracing');
// I am using Jaeger as exporter
const { JaegerExporter } = require('@opentelemetry/exporter-jaeger');
// This is not mandatory, by default httptrace propagation is used
// but it is not well supported by the PHP ecosystem and I have
// a PHP service to instrument. I discovered B3 is supported
// form all the languages I where intrumenting
const { B3Propagator } = require('@opentelemetry/core');
module.exports = (serviceName, jaegerHost, logger) => {
// A lot of those plugins are automatically loaded when you install them
// So if you do not use express for example you do not have to enable all
// those plugins manually. But Express is not auto enabled so I had to add them
// all
const provider = new NodeTracerProvider({
plugins: {
mongodb: {
enabled: true,
path: '@opentelemetry/plugin-mongodb',
},
http: {
enabled: true,
path: '@opentelemetry/plugin-http',
// I didn't do it in my example but it is a good idea to ignore health
// endpoint or others if you do not need to trace them.
ignoreIncomingPaths: [
'/',
'/health'
]
},
express: {
enabled: true,
path: '@opentelemetry/plugin-express',
},
}
});
// Here is where I configured the exporter, setting the service name
// and the jaeger host. The logger is helpful to track errors from the
// exporter itself
let exporter = new JaegerExporter({
logger: logger,
serviceName: serviceName,
host: jaegerHost
});
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register({
propagator: new B3Propagator(),
});
// Set the global tracer so you can retrieve it from everywhere else in the
// app
return opentelemetry.trace.getTracer("discount");
};
</code></pre>
<p>You will be thinking, that’s too easy! You are right, the nature of NodeJS
makes tracing very code agnostic. With this configuration you get a lot “for
free”.</p>
<p>You get a bunch of spans for every http request that ExpressJS serves, plus a
span for every MongoDB query. All of them with useful information like the
status code, path, user agents, query statements and so on.</p>
<p>We have to include it in our <code>./server.js</code> the entrypoint for our nodejs
application:</p>
<pre><code class="language-js">'use strict';
const url = process.env.DISCOUNT_MONGODB_URL || 'mongodb://discountdb:27017';
const jaegerHost = process.env.JAEGER_HOST || 'jaeger';
const logger = require('pino')()
// Import and initialize the tracer
const tracer = require('./tracer')('discount', jaegerHost, logger);
var express = require("express");
var app = express();
const MongoClient = require('mongodb').MongoClient;
const dbName = 'shopmany';
const client = new MongoClient(url, { useNewUrlParser: true });
const expressPino = require('express-pino-logger')({
logger: logger.child({"service": "httpd"})
})
</code></pre>
<p>As I told you, that’s it! With this code you have enough to make your NodeJS
application to show up in your trace.</p>
<p>The instrumented version of the application is available here
<a href="https://github.com/gianarb/shopmany/tree/discount/opentelemetry/discount">github.com/gianarb/shopmany/tree/discount/opentelemetry</a></p>
<h2 id="understand-the-project">understand the project</h2>
<p>I tend to checkout projects when in the process of learning how they work.
Documentation is useful but always incomplete for such a high moving projects.</p>
<p>I have to say that the scaffolding is clear even for an not fluent NodeJS
developer like me.</p>
<pre><code>$ tree -L 1
.
├── benchmark
├── CHANGELOG.md
├── codecov.yml
├── CONTRIBUTING.md
├── doc
├── examples
├── getting-started
├── karma.base.js
├── karma.webpack.js
├── lerna.json
├── LICENSE
├── package.json
├── packages
├── README.md
├── RELEASING.md
├── scripts
├── tslint.base.js
└── webpack.node-polyfills.js
</code></pre>
<p>I would like to define it as a monorepo, and it uses
<a href="https://github.com/lerna/lerna">lerna</a> to delivery multiple packages from the
same repository.</p>
<p><code>examples</code> contains workable example of how to use the different <code>packages</code>.</p>
<pre><code>$ tree -L 1 ./examples/
./examples/
├── basic-tracer-node
├── dns
├── express
├── grpc
├── grpc_dynamic_codegen
├── http
├── https
├── ioredis
├── metrics
├── mysql
├── opentracing-shim
├── postgres
├── prometheus
├── redis
└── tracer-web
$ tree -L 1 ./packages/
./packages/
├── opentelemetry-api
├── opentelemetry-base
├── opentelemetry-context-async-hooks
├── opentelemetry-context-base
├── opentelemetry-context-zone
├── opentelemetry-context-zone-peer-dep
├── opentelemetry-core
├── opentelemetry-exporter-collector
├── opentelemetry-exporter-jaeger
├── opentelemetry-exporter-prometheus
├── opentelemetry-exporter-zipkin
├── opentelemetry-metrics
├── opentelemetry-node
├── opentelemetry-plugin-dns
├── opentelemetry-plugin-document-load
├── opentelemetry-plugin-express
├── opentelemetry-plugin-grpc
├── opentelemetry-plugin-http
├── opentelemetry-plugin-https
├── opentelemetry-plugin-ioredis
├── opentelemetry-plugin-mongodb
├── opentelemetry-plugin-mysql
├── opentelemetry-plugin-postgres
├── opentelemetry-plugin-redis
├── opentelemetry-plugin-user-interaction
├── opentelemetry-plugin-xml-http-request
├── opentelemetry-propagator-jaeger
├── opentelemetry-resources
├── opentelemetry-shim-opentracing
├── opentelemetry-test-utils
├── opentelemetry-tracing
├── opentelemetry-web
└── tsconfig.base.json
</code></pre>
<p>The suffix of the package helps you to figure out what they are:</p>
<ul>
<li><code>opentelemetry-plugin-*</code> usually contains the code that instrument a specific
library, you can see here <code>express</code>, <code>http</code>, <code>https</code>, <code>dns</code>. Some plugins are
loaded by the <code>NodeTracerProvider</code> by default. Other has to be specified. You
can to relay on the code or read the documentation to figure it out. For
example <code>http</code> is loaded by default but if you need <code>express</code> you have to load
them up by yourself, figuring out the right dependencies. At least or now.</li>
<li><code>opentelemetry-exporter-*</code> contains various exporters for now Jaeger,
Prometheus, Zipkin the and otel-collector.</li>
</ul>
<p>Anyway, what I am trying to say is that it is very intuitive and looking here it
is clear what you can get from this project.</p>
<h2 id="plugin">Plugin</h2>
<p>NodeJS sounds very easy to instrument and on the right path to get automatic
instrumentation right, because you can listen from the outside to function
call, you do not need to specifically change your code where you do a request or
where you get one, you can add tracing in a centralized location. That’s how the
provided plugins work.</p>
<p><a href="https://github.com/othiym23/shimmer">Shimmer</a> is the library that simplify the
trick. I recently had a chat with <a href="https://twitter.com/walterdalmut">Walter</a>
because I know he works in NodeJS and during the experiments otel were easy
enough to fit his use case. He is currently trying it and as he discovered that
<a href="https://github.com/Automattic/mongoose">mongoose</a>, the ORM library he uses does
not use the officially provided <a href="https://mongodb.github.io/node-mongodb-native/">mongodb
driver</a> so the
otel-plugin-mongodb where not magically tracing his requests to mongodb, sadly.
But he is currently writing a <a href="https://github.com/wdalmut/opentelemetry-plugin-mongoose">plugin for
that</a>, so it won’t be
a problem pretty soon.</p>
Checklist for a new projectA person list I developed along those years that I try to implement across projects I start or contribute tohttps://gianarb.it/img/myselfie.jpg-large2020-04-02T09:08:27+00:002020-04-02T09:08:27+00:00https://gianarb.it/blog/new-project-checklist<p>Back in the day I used to start a lot of projects. From zero on GitHub, some of
them are still there, unused probably.</p>
<p>Recently I started to take part of other people projects like
<a href="https://github.com/testcontainers">testcontainers</a> or
<a href="https://github.com/profefe">profefe</a>. I wrote about why I do it during the
<a href="/blog/year-in-review">“2019 year in review”</a> post.</p>
<p>In both cases, so even when joining a new existing project or when I start a new
one I try to follow a checklist.</p>
<p>I developed this checklist along the years, moving parts and extending the
number of checks. The main goal for that is to validate that the project has
good answers for a couple of questions, not related at what it does, but to how
it does it.</p>
<ol>
<li>is it easy to onboard as a user?</li>
<li>as a new contributor is the project easy to understand?</li>
<li>as a maintainer do I have everything I can under control in order to waste as
less time as possible?</li>
</ol>
<p>I follow the checklist when working on opensource but also in close source
project and what I like about it is that you can propose a change by youself,
you can try to apply those feedback as a solo-developer, hoping to make
contributors, maintainers and colleagues to buy them spreading joy.</p>
<p>But let’s make to the list now.</p>
<h3 id="have-a-place-where-you-can-write">Have a place where you can write</h3>
<p>When I start a new project, but also during the onboard of an existing one in my
tool chain I look for a written format of it.</p>
<p>I look for a readme, an installation process, a getting started guide, a
contribution document. It does not need to be pretty one, a copy/paste of a few
bash scripts that the maintainer does to set itself up is enough.</p>
<p>Having a place during the early days of a project where I can write what I
think, how I would like to get things done is important in order to design
something usable and to spot sooner misleading assumption.</p>
<p>If you can build the place for all those information it will take you one
second to save them forever, it is just a matter of copy/pasting the command you
run in your terminal to spin up dependencies, build the project and so on.</p>
<p>I like to use the README.md, CONTRIBUTOR.md and a <code>./docs</code> folder to save
everything I am thinking about or everything I do that I hope will make my life
easy in a month where I will be back on that piece of code without even knowing
it was there. The feeling you get is the same a new person has when it looks at
your project for the first time.</p>
<p>There is no way you can get it right since the beginning, because there is not a
definition of right. First day everything you write is mainly for yourself, in a
month and some editing it will become the first version of the documentation for
your project.</p>
<h3 id="logging-and-instrumentation-library">Logging and instrumentation library</h3>
<p>As I said at the beginning of the article all the checks do not depend on the
business logic of your application or library. All of them has to speak with the
outside world sharing their internal state in a way that is reusable,
comprehensive, configurable.</p>
<p>There are a lot of people that speaks about observability, logging, tracing,
monitoring. Everybody has its own opinion, but form a technical point of your
what you write has to be easy to troubleshoot.</p>
<p>You do it using the right telemetry libraries. For logging I do not have any
doubt. In Go I use <a href="https://github.com/uber-go/zap">zap</a>.</p>
<p>During a workshop about observability I built where I had to instrument 4
applications on different languages I selected:</p>
<ul>
<li><a href="https://github.com/pinojs/pino">pino</a> for NodeJS</li>
<li><a href="https://github.com/Seldaek/monolog">monolog</a> for PHP</li>
<li><a href="https://logging.apache.org/log4j/2.x/">log4j</a> for JAVA</li>
</ul>
<p>In general I look for libraries that allow me to do structured logging, so for
the one that enables me to attach key value pairs to log line. I also look for
logging libraries that has the concept of exporters and format. Nothing unusual.</p>
<p>For tracing and events I do not have a favourite one but I would like to see
<a href="https://opentelemetry.io">Opentelemetry</a> to become the way to go.</p>
<h3 id="continuous-integration">Continuous integration</h3>
<p>A project without CI can not be called in that way. Nowadays there are a lot of
free services that you can use, so no excuse. When I am on GitHub I go for
Actions now because they are free and embedded in the VCS itself.</p>
<p>If you didn’t write any test at least set the process up and running. Just run
tests, usually they do not fails if empty. And there are static checker, linters
and things like that for every language, set them up!</p>
<h3 id="continuous-delivery">Continuous delivery</h3>
<p>You made the CI part, you are half way done. Release is important and we have
the tools to get it right since day one. It is a pain to do a release, there are
a lot of potential manual state to get it right:</p>
<ol>
<li>Bump version</li>
<li>Changelog</li>
<li>Compile and push binaries if it is an application</li>
<li>…</li>
</ol>
<p>There are tools that helps you to do that in automation. For my apps I use
<a href="https://github.com/goreleaser/goreleaser">goreleaser</a>, for the libraries I use
<a href="https://github.com/marketplace/actions/release-drafter">Releaser drafter</a>.</p>
<h3 id="testing-framework">Testing framework</h3>
<p>Write tests, and when you see repeated code extract it in a testing package.
<code>zap</code> has <code>zaptest</code>, your project should have <code>yourprojecttest</code> as well.</p>
<p>It is useful for yourself, because it will make the job to write more test
effortless, and if you well document your testing package contributors will be
able to use it when opening a PR because you will make writing tests easier for
everybody. As a bonus who ever uses your libraries can use the testing package
to write their own tests for their application.</p>
<h2 id="conclusion">Conclusion</h2>
<p>This is the list I use, and I will keep it up to date now that I wrote it down,
adding or editing it, so be sure to stay around!</p>
<p>I hope this checklist is general enough and useful to be reusable for you in
some of its part.</p>
<p>What I like about it is that I do not need to be a CTO, a maintainer or
something like that to drive the adoption of those point that I think are
crucial, I drove the adoption of some of them even as a solo contributor.</p>
Why code instrumentation?I decided to finally create a category about code instrumentation. Because I am a develop. And I think it matters. It is important to write better code and more reliability application that we can learn from.https://gianarb.it/img/got-your-back.jpg2020-03-29T09:08:27+00:002020-03-29T09:08:27+00:00https://gianarb.it/blog/why-code-instrumentation<p>I am writing this blog post as a common introduction for a new category I would
like to write about consistently on my blog. If this is the first time you land
here this is my blog, and I write about everything that catches my attention but
sometime earlier sometime later on I realize I can group my posts in categories,
and that’s what I am doing now.</p>
<p>Some of them are: <a href="/planet/assemble-kubernetes.html">Assemble Kubernetes</a>,
<a href="/planet/docker.html">Docker</a>, <a href="/planet/mockmania.html">MockMania</a>. This one
will be called <code>Code Instrumentation</code>.</p>
<p>There are a lot of people writing about observability, monitoring and I did it
for the last 3 years as well. I learned a lot along the way but what I think is
crucial is that developers has to write code that is
understandable and easy to debug where it is more valuable, in production. And
if an application or a system is hard to figure out we as a developer play a
mojor role on it.</p>
<p>That’s why Site Reliability Engineering (SRE) is not related to ops, servers,
Kubernetes but it is something that plays its match in your code.</p>
<p>What’s why I think SRE and DevOps are different, not at all connected.</p>
<p>The technologies that are leading the landscape are:</p>
<ol>
<li>Prometheus but not the time series database, their client libraries and the
exposition format, now branded from the community and the Cloud Native
Computing Foundation (CNCF) as OpenMetrics</li>
<li>OpenTracing, OpenCensus and OpenTelemetry. They are part of the same bullet
points because I think about them as the consequence of each other since what
I hope is “THE LAST ONE”, OpenTelemetry. They are instrumentation libraries
and specification to increase interoperability and to avoid vendor lock-in
for what concerns distributed tracing and metrics. I hope logs will jump
onboard at some point</li>
</ol>
<h2 id="prometheus-and-openmetrics">Prometheus and OpenMetrics</h2>
<p>I wrote about this topic previously, so have a look there if you do not know
what I am speaking about.</p>
<p>I think they are worth to mention here because that’s how I learned the effect
of good or bad code instrumentation, and the fact that it has to happen in your
code, when you develop it.</p>
<p>It has the same weight has writing a good data structure, writing solid unit
tests, or picking the right design pattern.</p>
<h2 id="opentelemetry-otel">OpenTelemetry (otel)</h2>
<p>As I said I will refer to otel when I can, not because I think OpenTracing or
OpenCensus is bad, but because I do not see this as a religion, but for me it is
a technical problem, they is well spread and it has to find a good answer.</p>
<p>Those communities decided to merge to otel in the way they are doing, good or
bad? We can get a beer at some point and I will tell you. It is out of scope.</p>
<h2 id="what-i-am-gonna-talk-about">What I am gonna talk about</h2>
<p>This is a long new category introduction blog post probably but that’s it. Over
the last two years I tried to share what I experienced around this topic with a
workshop called: “Application Monitoring”. A lot of the articles that I will
write comes from there, and it is an attempt to share what I think worked or
failed.</p>
<h2 id="links">Links</h2>
<ul>
<li><a href="/planet/code-instrumentation.html">All about Code Instrumentation</a> from my blog</li>
<li><a href="https://github.com/gianarb/shopmany">ShopMany</a> is the application I developed for the workshop</li>
<li><a href="https://github.com/gianarb/workshop-observability">Workshop notes</a> contains notes, exercises and solutions for the lessons I
proposed in the workshop itself</li>
<li><a href="https://www.honeycomb.io/blog/">honeycomb</a> because when you speak about o11y you have to quote them!</li>
<li><a href="/tinyletter.html">My newsletter</a> is probably the best way to stay in touch with the content I
create</li>
<li><a href="https://twitter.com/gianarb">Twitter</a> is the best way to stay in touch with me</li>
</ul>
CNCF Webinar: Continuous Profiling Go Application Running in KubernetesSlides, videos and links from a webinar I have with the CNCF about kubrenetes, profefe, golang and pprof.https://gianarb.it/img/cncf-logo.png2020-03-27T09:08:27+00:002020-03-27T09:08:27+00:00https://gianarb.it/blog/cncf-webinar-kubernetes-pprof-profefe<div class="embed-responsive embed-responsive-16by9">
<iframe class="embed-responsive-item" src="https://www.youtube.com/embed/SzhQZQ6VGoY" allowfullscreen=""></iframe>
</div>
<p>Microservices and Kubernetes help our architecture to scale and to be
independent at the price of running many more applications. Golang provides a
powerful profiling tool called pprof, it is useful to collect information from a
running binary for future investigation. The problem is that you are not always
there to take a profile when needed, sometimes you do not even know when you
need to one, that’s how a continuous profiling strategy helps. Profefe is an
open-source project that collect and organizes profiles. Gianluca wrote a
project called kube-profefe to integrate Kubernetes with Profefe. Kube-profefe
contains a kubectl plugin to capture locally or on profefe profiles from running
pods in Kubernetes. It also provides an operator to discover and continuously
profile applications running inside Pods.</p>
<p>A bunch of links for you:</p>
<ul>
<li>Video coming soon</li>
<li><a href="/blog/go-continuous-profiling-profefe">My article: Continuous profiling in Go with Profefe</a></li>
<li><a href="/blog/continuous-profiling-go-apps-in-kubernetes">My article: Continuous Profiling Go applications running in Kubernetes</a></li>
<li><a href="https://research.google/pubs/pub36575/">Google-Wide Profiling: A Continuous Profiling Infrastructure for Data Centers</a></li>
<li><a href="https://github.com/profefe/profefe">Profefe on Github</a></li>
<li><a href="https://github.com/profefe/kube-profefe">Kube Profefe on Github</a></li>
<li><a href="https://github.com/google/pprof">google/pprof</a> library on GitHub</li>
<li><a href="https://kubernetes.profefe.dev">Work in progress documentation! help me out!</a></li>
</ul>
<div class="embed-responsive embed-responsive-16by9">
<iframe class="embed-responsive-item" src="//speakerdeck.com/player/ff55e041659945bca5d31013bd999c28" allowfullscreen=""></iframe>
</div>
How to do testing with zap a popular logging library for GolangHow you can use logging to build assertions when testing. What the popular Golang logging library provided by Uber gives you around unit tests.https://gianarb.it/img/golang-mockmania.png2020-03-24T09:08:27+00:002020-03-24T09:08:27+00:00https://gianarb.it/blog/golang-mockmania-zap-logger<div class="alert alert-dark" role="alert">
<div class="row">
<div class="col-md-2 align-self-center">
<a href="https://link.testproject.io/0ak" target="_blank">
<img class="img-fluid" src="/img/testproject-logo-small.png" />
</a>
</div>
<div class="col-md-8 text-center">
<a href="https://link.testproject.io/0ak" class="alert-link" target="_blank">TestProject</a> is a community all
about testing and you know how much I love communities! Join us.
</div>
</div>
</div>
<p>If you follow me on <a href="https://twitter.com/gianarb">twitter</a> you know that
I am passionate about o11y, monitoring and code instrumentation.</p>
<p>I see logs not as a random print statement that you use only when something is
wrong, but they have value. Logs are the communication channel our applications
use. As developers it is our job to make them to speak in a comprehensive way.</p>
<p>Logs should be structured and in some way consistent across functions, http
handlers, applications even languages to simplify their use. From algorithms and
human operators.</p>
<p>In Go <a href="https://github.com/uber-go/zap">zap</a> is a popular logging library provided by Uber, I
use it almost by default for all my applications.</p>
<pre><code class="language-go">package main
import "go.uber.org/zap"
func main() {
logger, _ := zap.NewProduction()
do(logger)
}
func do(logger *zap.Logger) {
logger.Error("Start doing things")
}
</code></pre>
<p>So logging and testing? In the same article? I should be really drunk!</p>
<p class="text-center"><img src="/img/kermit-frog-drunk.jpg" alt="" class="img-fluid" /></p>
<p>When I discovered that <code>zap</code> comes with a testing utility package called
<code>zaptest</code> I felt in love with this library even more:</p>
<pre><code class="language-go">package main
import (
"testing"
"go.uber.org/zap/zaptest"
)
func Test_do(t *testing.T) {
logger := zaptest.NewLogger(t)
do(logger)
}
</code></pre>
<p>The <code>go test</code> command supports the flag <code>-v</code> to improve verbosity of test
execution. In practice that’s how you forward to <code>stdout</code> logs and print
statements during a test execution. <code>zaptest</code> works with that as well.</p>
<p>Very cool, and useful if you write smoke tests, pipeline tests, or how ever you
call them and see the logs can be spammy, but helpful to figure out the actual
issue.</p>
<pre><code class="language-go">package main
import (
"testing"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"go.uber.org/zap/zaptest"
)
func Test_do(t *testing.T) {
logger := zaptest.NewLogger(t, zaptest.WrapOptions(zap.Hooks(func(e zapcore.Entry) error {
if e.Level == zap.ErrorLevel {
t.Fatal("Error should never happen!")
}
return nil
})))
do(logger)
}
</code></pre>
<p>You can use <code>hooks</code> to check for expected or unexpected logs.</p>
<p>Hook are executed for every log line:</p>
<pre><code class="language-go">func(e zapcore.Entry) error {
if e.Level == zap.ErrorLevel {
t.Fatal("Error should never happen!")
}
return nil
})
</code></pre>
<p>If you do not expect any error level log line for your execution because you are
testing the happy path, you can do something like that.</p>
<p><strong>DISCALIMER:</strong> This is another way to write assertion. You will may use them to enforce
other checks, or to validate the workflow from a different point of view that
will may be easier to do as first attempt. As I usually say: “an easy and
partial test is better than no test”.</p>
<p>Do not test only logs, it won’t age well! Keep writing good tests!</p>
Show Me Your Code with Walter Dal Mut: Extend Kubernetes in NodeJSLet's try to get virtual! This is the first attempt as CNCF Meetup from Turin to do something online! The series is called Show me your code. Walter dal Mut from Corley will be the guinea pig to test this new format. Live show on YouTube about Kubernetes and how to use shared informer to extend its capabilities in Node.js.https://gianarb.it/img/show-me-your-code/ep1-thump.jpg2020-03-13T09:08:27+00:002020-03-13T09:08:27+00:00https://gianarb.it/blog/show-me-your-code-walter-dal-mut-kubernetes-nodejs-informers<h2 id="about-walter-dal-mut">About Walter Dal Mut</h2>
<p>Walter Dal Mut works as a Solutions Architect <a href="https://corley.it/">@Corley SRL</a>,
is is an electronic engineer who moved to Software Engineering and Cloud
Computing Infrastructures. Passionate about technology in general and open
source movement lover.</p>
<p>If you wont you can follow him on <a href="https://twitter.com/walterdalmut">Twitter</a>
and <a href="https://github.com/wdalmut">GitHub</a>.</p>
<h2 id="kubernetes-extendibility-and-nodejs">Kubernetes extendibility and NodeJS</h2>
<p>Almost everybody that is currently working on Kubernetes, developing extensions,
controllers, operator is doing it in Go. That’s reasonable because Kubernetes is
written Go and there is a lot of code that you can reuse in that language.</p>
<p>What if you are not a Go developer?</p>
<p>Walter coded a shared informer in NodeJS that watches and take actions on Pod
events.</p>
<h2 id="links">Links</h2>
<ul>
<li>The code you saw in the video lives here
<a href="https://github.com/wdalmut/k8s-informer-ytlive">wdalmut/k8s-informer-ytlive</a></li>
<li><a href="https://get.oreilly.com/ind_extending-kubernetes.html">Extend Kubernetes O’Reilly report</a></li>
<li><a href="https://engineering.bitnami.com/articles/a-deep-dive-into-kubernetes-controllers.html">A deep dive into Kubernetes
controllers</a></li>
</ul>
How to test CLI commands made with Go and CobraCLI commands are common in Go. Testing them is an effective way to run a big amount of code that is actually very close to the end user. I use Cobra, pflags and Viper and that's what I do when I write unit test for Cobra commandshttps://gianarb.it/img/golang-mockmania.png2020-03-09T09:08:27+00:002020-03-09T09:08:27+00:00https://gianarb.it/blog/golang-mockmania-cli-command-with-cobra<p>Almost everything is a CLI application when writing Go. At least for me. Even
when I write an HTTP daemon I still have to design a UX for configuration
injection, environment variables, flags and things like that.</p>
<p>The set of libraries I use is very standard, I use
<a href="https://github.com/spf13/cobra">Cobra</a>,
<a href="https://github.com/spf13/pflag">pflags</a> and occasionally
<a href="https://github.com/spf13/viper">Viper</a>. I can say, without a doubt that <a href="https://twitter.com/spf13">Steve
Francia</a> is awesome!</p>
<p>This is how a command looks like directly from the Cobra documentation:</p>
<pre><code>var rootCmd = &cobra.Command{
Use: "hugo",
Short: "Hugo is a very fast static site generator",
Long: `A Fast and Flexible Static Site Generator built with
love by spf13 and friends in Go.
Complete documentation is available at http://hugo.spf13.com`,
Run: func(cmd *cobra.Command, args []string) {
// Do Stuff Here
},
}
</code></pre>
<p>I like to write a constructor function that returns a command, in this case it
will be something like:</p>
<pre><code>func NewRootCmd() *cobra.Command {
return &cobra.Command{
Use: "hugo",
Short: "Hugo is a very fast static site generator",
Long: `A Fast and Flexible Static Site Generator built with
love by spf13 and friends in Go.
Complete documentation is available at http://hugo.spf13.com`,
Run: func(cmd *cobra.Command, args []string) {
// Do Stuff Here
},
}
}
</code></pre>
<p>The reason why I like the have this function is because it helps me to clearly
see the dependency my command requires. In this case nothing. I also like to use
not the Run function but the RunE one, it works in the same way but it expects
an error in return.</p>
<pre><code>func NewRootCmd(in string) *cobra.Command {
return &cobra.Command{
Use: "hugo",
Short: "Hugo is a very fast static site generator",
Long: `A Fast and Flexible Static Site Generator built with
love by spf13 and friends in Go.
Complete documentation is available at http://hugo.spf13.com`,
RunE: func(cmd *cobra.Command, args []string) (error) {
fmt.Fprintf(cmd.OutOrStdout(), in)
return nil
},
}
}
</code></pre>
<p>In order to execute the command, I use cmd.Execute().</p>
<p>Let’s write a test function:</p>
<p>The output with <code>go test -v</code> contains “hi” because by default cobra prints to
stdout, but we can replace it to assert that automatically</p>
<pre><code>func Test_ExecuteCommand(t *testing.T) {
cmd := NewRootCmd("hi")
cmd.Execute()
}
</code></pre>
<pre><code>=== RUN Test_ExecuteCommand
hi--- PASS: Test_ExecuteCommand (0.00s)
PASS
ok ciao 0.006s
</code></pre>
<p>The trick here is to replace the stdout with something that we can read
programmatically like a bytes.Buffer for example:</p>
<pre><code class="language-go">func Test_ExecuteCommand(t *testing.T) {
cmd := NewRootCmd("hi")
b := bytes.NewBufferString("")
cmd.SetOut(b)
cmd.Execute()
out, err := ioutil.ReadAll(b)
if err != nil {
t.Fatal(err)
}
if string(out) != "hi" {
t.Fatalf("expected \"%s\" got \"%s\"", "hi", string(out))
}
}
</code></pre>
<p>Personally I do not think there is much more to know in order to effectively
test CLI commands, they can be very complex, but if you can mock its
dependencies and check what the execution prints out you are very flexible!</p>
<p>Another thing you have to control when running a command is its arguments and
its flags because based on them you will get different behavior that you have to
test in order to figure out that your commands work with all of them.</p>
<p>The logic works the same for both but arguments are very easy, you just have to
set the argument in the command with the function
<code>cmd.SetArgs([]string{"hello-by-args"}).</code></p>
<pre><code class="language-go">package main
import (
"bytes"
"fmt"
"io/ioutil"
"testing"
"github.com/spf13/cobra"
)
func NewRootCmd() *cobra.Command {
return &cobra.Command{
Use: "hugo",
Short: "Hugo is a very fast static site generator",
RunE: func(cmd *cobra.Command, args []string) error {
fmt.Fprintf(cmd.OutOrStdout(), args[0])
return nil
},
}
}
func Test_ExecuteCommand(t *testing.T) {
cmd := NewRootCmd()
b := bytes.NewBufferString("")
cmd.SetOut(b)
cmd.SetArgs([]string{"hi-via-args"})
cmd.Execute()
out, err := ioutil.ReadAll(b)
if err != nil {
t.Fatal(err)
}
if string(out) != "hi-via-args" {
t.Fatalf("expected \"%s\" got \"%s\"", "hi-via-args", string(out))
}
}
</code></pre>
<p>Flags works in the same:</p>
<pre><code class="language-go">package main
import (
"bytes"
"fmt"
"io/ioutil"
"testing"
"github.com/spf13/cobra"
)
var in string
func NewRootCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "hugo",
Short: "Hugo is a very fast static site generator",
RunE: func(cmd *cobra.Command, args []string) error {
fmt.Fprintf(cmd.OutOrStdout(), in)
return nil
},
}
cmd.Flags().StringVar(&in, "in", "", "This is a very important input.")
return cmd
}
func Test_ExecuteCommand(t *testing.T) {
cmd := NewRootCmd()
b := bytes.NewBufferString("")
cmd.SetOut(b)
cmd.SetArgs([]string{"--in", "testisawesome"})
cmd.Execute()
out, err := ioutil.ReadAll(b)
if err != nil {
t.Fatal(err)
}
if string(out) != "testisawesome" {
t.Fatalf("expected \"%s\" got \"%s\"", "testisawesome", string(out))
}
}
</code></pre>
<p>This is it! I like a lot to write unit tests for cli command because in real
life they are way more complex than the one I used here. It means that they run
a lot more functions but the command is well scoped in terms of dependencies (if
you write a constructor function) and in terms of input and output. So it is
easy to write an assertion and write table tests with different inputs.</p>
Smart working does not need to be remoteThere is a different between remote work and smart work. You can have both, or just one. It is on you. I prefer both at the moment!https://gianarb.it/img/gianarb.png2020-03-04T09:08:27+00:002020-03-04T09:08:27+00:00https://gianarb.it/blog/smart-working-does-not-need-to-be-remote<p>I work remotely for almost 3 years now and I am happy with it. First of all,
because the company I am working for, InfluxData, is not based where I live. I
am currently in Turin and it is in San Francisco, so the unique way for me to do
the kind of work I am doing today is to be remote. I moved to Dublin for 2 years
because my English was not as I hoped it to be and I was looking for a quick way
to make it right. Beers and good friends helped me to succeed! I still have to
use Grammarly to write blog posts but hey, at least I can write them.</p>
<p>Anyway, I like to work with people from all over, and that’s why I am not even
thinking about the possibility to get back to work for a small company in Italy
for now. Right now the perception is almost like getting paid to work in the
same environment I used to work for free when contributing to open source
communities. People from all over my conditions in terms of time commitment.
I just have to work hard and do what it takes to make my team and the
company to improve. This is, in essence, the difference between remote work
and smart working. You can work remotely as you work in an office. You have
to be at your desk, at some time for 8 hours, even when it is raining, even
when you feel unproductive.</p>
<p>This rambling is to remember to myself that I do not like remote working, I like
smart working. I like to be able to organize my time because my boss trusts my
ability to judge the situation. I can work day and night and take a break an
entire day if I feel like I have to. Obviously it is a huge responsibility and a
risk from both side, as a company you do not have the common framework that we
built across years to figure out how productive an employee is, and as a worker,
you have to develop the risk skills to read the situation you are in. But this
ability helped me to be a conscious developer and not just a code generator that
solves self-built challenges.</p>
<p class="text-center"><img src="/img/cruna-ago.jpg" alt="Middle East, Needle, Threads, Sewing Thread" class="img-fluid" /></p>
<p class="small text-center">Hero image via <a href="https://pixabay.com/photos/middle-east-needle-threads-4854847/">Pixabay</a></p>
<p>So I would like to rephrase the title: “Smart working does not need to be
remote, but it helps”.</p>
<p>I realized the difference because I had to get smarter, my company is 9 hours
behind my current time. I work alone a lot. I have to read in advance what the
product or my product manager will ask me to work on because I have to develop a
buffer that will keep you busy when the current task I am resolving is blocked
or I can’t get around it without reaching to a person that will probably be
offline, or maybe I can but it will take so much effort that it is smarter for
me to just wait. It is equivalent to procrastinate until you can shake the chair
of your coworker that wrote the freaking recursive function that you have to
debug. The problem is that you never know if the coworker will even show up.</p>
<p>Companies you have to set up your workload to assist smart workers, not a remote
worker. In tech at least, where this is a reachable goal.</p>
<p>I think it is a temporary condition, right now I feel like I need both, remote
to be able to work where I think I will learn or perform better, or where it is
funnier. And smart, because I like to develop organization skills and to feel
the master of my clock.</p>
<p>Who knows how it will evolve.
Thank you for your time.</p>
The awesomeness of the httptest package in GoOne of the reasons why testing in Go is friendly is driven by the fact that the core team already provides useful testing package as part of the stdlib that you can use, as they do to test packages that depend on them. This article explains how to use the httptest package to mock HTTP servers and to test sdks that use the http.Client.https://gianarb.it/img/golang-mockmania.png2020-02-25T09:08:27+00:002020-02-25T09:08:27+00:00https://gianarb.it/blog/golang-mockmania-httptest<p>Go has a nice http package. I am able to say that because I am not aware of any
other implementation of it in Go other than the one provided by the standard
library. This is for me a good sign.</p>
<pre><code class="language-go">resp, err := http.Get("http://example.com/")
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
</code></pre>
<p>This example comes from the <a href="https://golang.org/pkg/net/http/">documentation</a>
itself.</p>
<p>We are here to read about testing, so who cares about the http package itself!
What matters is the <a href="https://golang.org/pkg/net/http/httptest/">httptest</a>
package! Way cooler.</p>
<p>This article is not the first one for the MockMania series, I wrote about titled
<a href="https://gianarb.it/blog/golang-mockmania-influxdb-v2-client">“InfluxDB Client
v2”</a>, it uses the
httptest service already! But hey it deserves its own blog post.</p>
<h2 id="server-side">Server Side</h2>
<p>The http package provides a client and a server. The server is made of handlers.
The handler takes a request and based on that it returns a response. This is its
interface:</p>
<pre><code class="language-go">type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
</code></pre>
<p>As you can see if gets a ResponseWriter to compose a response based on the
Request it gets. This process can be as complicated as you like, it can reaches
databases, third party services but in the end, it writes a response.</p>
<p>It means that mocking all the dependencies to get the right scenario we use the
ResponseWriter to figure out if the handler made what we want.</p>
<p>The httptest package provides a replacement for the ResponseWriter called
ResponseRecorder. We can pass it to the handler and check how it looks like
after its execution:</p>
<pre><code class="language-go">handler := func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "ping")
}
req := httptest.NewRequest("GET", "http://example.com/foo", nil)
w := httptest.NewRecorder()
handler(w, req)
resp := w.Result()
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(resp.StatusCode)
fmt.Println(string(body))
</code></pre>
<p>This handler is very simple, it just manipulates the response body. If your
handler is more complicated and it has dependencies you have to be sure to
replace them as well, injecting the appropriate one.</p>
<h2 id="client-side">Client-Side</h2>
<p>Handlers are useful if you can’t use them. The Go http package provides an http
client as well that you can use to interact with an http server. An http client
by itself is useless, but it is the entry point for all the manipulation and
transformation you do on the information you get via HTTP. With the
proliferation of microservices, it is a very common situation.</p>
<p>The workflow is well understood, you have an HTTP backend to interact with, you
fetch data from there are you manipulate them with your business logic. When
testing what you can do is to mock the http backend in order to return what you
want, testing that your business logic does what it is supposed to do based on
the input you get from the HTTP server.</p>
<p>During our first example, the handler was the subject of our testing, this is
not the case anymore, we are testing the consumer this time, so we have to mimic
and handler in order to get what we expect to return</p>
<pre><code class="language-go">ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "I am a super server")
}))
defer ts.Close()
</code></pre>
<p>As you can see we are creating a new HTTP server via the httptest. It accepts a
handler. The goal for this handler returns what we would like to gest our code
on. In theory, it should just use the ResponseWriter to compose the response we
expect.</p>
<p>The server has a bunch of methods, the one you are looking for is the URL one.
Because we can pass it to an http.Client, the one we will use as a mock for our
function</p>
<pre><code class="language-go">res, err := http.Get(ts.URL)
if err != nil {
log.Fatal(err)
}
bb, err := ioutil.ReadAll(res.Body)
res.Body.Close()
</code></pre>
<p>That’s it, as you can see <code>ts.URL</code> points the http.Client to the mock server we
created.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I use the httptest package a lot even when writing SDKs for services that do not
have integration with Go because I can follow their documentation mocking their
server and I do not need to reach them until I am confident with the code I
wrote.</p>
<p>My suggestion is to test your client code for edge cases as well because of the
httptest.Server gives you the flexibility to write any response you can think
about. You can mimic an authorized response to seeing how your code with handle
it, or an empty body or a rate limit. The only limit is our laziness.</p>
Golang MockMania InfluxDB Client v2https://gianarb.it/img/golang-mockmania.png2020-02-09T09:08:27+00:002020-02-09T09:08:27+00:00https://gianarb.it/blog/golang-mockmania-influxdb-v2-client<p>Recently I had to develop an integration with the <a href="https://github.com/influxdata/influxdb-client-go">InfluxDB Client v2 Golang
SDK</a>.</p>
<p>This SDK is useful to interact with InfluxDB v2, create organizations and users,
write new points, and submit queries; it accepts the Golang http.Client.</p>
<pre><code class="language-golang">influx, err := influxdb.New(myHTTPInfluxAddress, myToken, influxdb.WithHTTPClient(myHTTPClient))
if err != nil {
panic(err)
}
</code></pre>
<p>Having the ability to pass the HTTP client from the outside
<code>influxdb.WithHTTPClient(myHTTPClient)</code> improves the familiarity golang
developers have with the library; they know how to configure Transporters or how
to inject logging, tracing, debugging. For what concerns <code>Golang MockMania</code>, it
gives to use the possibility to pass the
<a href="https://golang.org/pkg/net/http/httptest/#example_Server">httptest</a> client.</p>
<pre><code class="language-golang">influxDBServer := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
}))
influxClient, err := influxdb.New(myHTTPInfluxAddress, myToken, influxdb.WithHTTPClient(influxDBServer.Client()))
</code></pre>
<p>At this point you can write the response you expect from the influxdb server
using the <code>http.ResponseWriter</code>.</p>
<p>Either way, even if you have to check what influxdb receives from the sdk or if
you have to obtain a specific answer from InfluxDB to validate what your
business logic will do, nothing will stop you from using checking the
http.Request or utilizing the http.ResponseWriter to get what you expect.</p>
Continuous Profiling Go applications running in KubernetesKube-Profefe is an open source project that acts like a bridge between Kubernetes and Profefe. It helps you to implement continuous profiling for Go applications running in Kubernetes.https://gianarb.it/img/profefe.png2020-02-04T09:08:27+00:002020-02-04T09:08:27+00:00https://gianarb.it/blog/continuous-profiling-go-apps-in-kubernetes<p>Recently I wrote <a href="https://gianarb.it/blog/go-continuous-profiling-profefe">“Continuous profiling in Go with
Profefe”</a>, an article
about the new shiny open source project I am contributing to.</p>
<p><strong>TLDR:</strong> Profefe is a registry for pprof profiles. You can push them embedding
an SDK in your application or you can write a collector (cronjob) that gets
profiles and push the tar via the Profefe API. Side by side with the profile you
have to send other information like:</p>
<ul>
<li>Type: represents the profile type such as mutex, goroutines, CPU and so on</li>
<li>Service: identifies the source for this profile, for example, the binary name</li>
<li>InstanceID: identifies where it comes from, for example, pod name or server
hostname</li>
<li>Labels: are optional key/value pairs that you can use at query time to filter
profiles. If you are building the same service with two different Go versions
to check for performance degradation you can label the profiles with
<code>go=1.13.4</code> for example.</li>
</ul>
<p>The article has way more content but that’s enough. You can keep reading with
only this information.</p>
<h2 id="kubernetes">Kubernetes</h2>
<p>As you know at InfluxData we use Kubernetes, our services already expose the
<a href="https://golang.org/pkg/net/http/pprof/">pprof HTTP handler</a> and we can not
instrument all the services with the Profefe SDK, for those reasons we had to
write our own collectors capable of getting pprof profiles via the Kubernetes
API and to push them into Profefe. That’s why we decided to go with a different
approach. I wrote a project called
<a href="https://github.com/profefe/kube-profefe">kube-profefe</a>. It acts as a bridge
between the Profefe API and Kubernetes. The repository provides two different
binaries:</p>
<ul>
<li>A kubectl plugin that you can install (even via krew) that servers useful
utilities to interact with the profefe API (profefe at the moment does not
have a CLI) and to capture profiles from running pod.</li>
<li>A collector that can run as a cronjob, it goes pod by pod looking for profiles
to collect and it will push them to Profefe.</li>
</ul>
<h2 id="architecture">Architecture</h2>
<p>In order to configure the collector or to capture profiles from a running
container, it leverages pod annotations. Only the pods with the annotation
<code>pprof.com/enable=true</code> will be taken into consideration from kube-profefe.
Other annotations are optional or they have default values. This one is the
unique one that has to be set to make kube-profefe aware of your pod.</p>
<p>The example above shows a Pod spec that enables profefe capabilities:</p>
<pre><code class="language-yaml">apiVersion: v1
kind: Pod
metadata:
name: influxdb-v2
annotations:
"profefe.com/enable": "true"
"profefe.com/port": "9999"
spec:
containers:
- name: influxdb
image: quay.io/influxdb/influxdb:2.0.0-alpha
ports:
- containerPort: 9999
</code></pre>
<p>As you can see there are other annotations such as <code>profefe.com/port</code> by default
is 6060. In this case it is pointed to 9999 because that’s where the pprof HTTP
handler runs in InfluxDB v2. A full list of annotations is maintained in the
project’s README.md.</p>
<p>There is not a lot more to know about the underling mechanism that enpowers
kube-profefe, we are gonna deep dive on both components: the kubectl plugin and
the collector.</p>
<h2 id="kubectl-profefe-the-kubectl-plugin">Kubectl-profefe: the kubectl plugin</h2>
<p>A kubectl plugin is nothing more than a binary located in your $PATH with the
prefix name “kubectl-”. In my case the binary is released with the name
kubectl-profefe, when located in your $PATH you will be able to run a command
like:</p>
<pre><code class="language-bash">$ kubectl profefe --help
It is a kubectl plugin that you can use to retrieve and manage profiles in Go.
Usage:
kubectl-profefe [flags]
kubectl-profefe [command]
Available Commands:
capture Capture gathers profiles for a pod or a set of them. If can filter by namespace and via label selector.
get Display one or many resources
help Help about any command
load Load a profile you have locally to profefe
Flags:
-A, --all-namespaces If present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace.
--as string Username to impersonate for the operation
--as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
--cache-dir string Default HTTP cache directory (default "/home/gianarb/.kube/http-cache")
--certificate-authority string Path to a cert file for the certificate authority
--client-certificate string Path to a client certificate file for TLS
--client-key string Path to a client key file for TLS
--cluster string The name of the kubeconfig cluster to use
--context string The name of the kubeconfig context to use
-f, --filename strings identifying the resource.
-h, --help help for kubectl-profefe
--insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
--kubeconfig string Path to the kubeconfig file to use for CLI requests.
-n, --namespace string If present, the namespace scope for this CLI request
-R, --recursive Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory. (default true)
--request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
-l, --selector string Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
-s, --server string The address and port of the Kubernetes API server
--token string Bearer token for authentication to the API server
--user string The name of the kubeconfig user to use
Use "kubectl-profefe [command] --help" for more information about a command.
</code></pre>
<p>This output should look very familiar to you, there are a lot of options usable
with any other kubectl native command. Mainly around authentication: –user,
–server, –kubeconfig, –client-certificate… Or around pod selection: -l,
–selector, -n, –namespace, –all-namespaces. If you are curious about how to
write a friendly kubectl plugin I wrote <a href="https://gianarb.it/blog/kubectl-flags-in-your-plugin">“kubectl flags in your
plugin”</a> check it out.</p>
<p>This plugin, even if it is not native, uses the same authentication mechanism in
use from the kubectl so, where ever the kubectl works, this plugin should work
as well.</p>
<p>The pod selectors -l, -n, for example, are useful when running the command:</p>
<pre><code>$ kubectl profefe capture
</code></pre>
<p>Capture, as the name suggests, goes straight to one or more pods and it
downloads or pushes to profefe various profiles. It is very flexible, you can
capture pprof profiles from a specific pod (or multiple pods) by ID:</p>
<pre><code>$ kubectl profefe capture <pod-id>,<pod-id>...
</code></pre>
<p><em>NB: just remember to use the namespace where the pods are running with the flag
-n or –namespace.</em></p>
<p>You can use the pod selectors to collect multiple profiles:</p>
<pre><code>$ kubectl profefe capture -n web
</code></pre>
<p>Captures profiles from all the pod with the pprof.com/enable=true annotation
running in the pod namespace and it will store them under the <code>/tmp</code> directory.
You can change the output directory with <code>--output-dir</code>. If you do not want to
store them locally you can push them to profefe specifying its location via
<code>--profefe-hostport</code>.</p>
<p>The are other combinations for the capture command and you can get profiles from
profefe, I will leave the rest to you!</p>
<p class="text-center"><img src="/img/stopwatch.jpg" alt="" class="img-fluid" /></p>
<p class="small text-center">Hero image via <a href="https://pixabay.com/illustrations/time-time-management-stopwatch-3216244/">Pixabay</a></p>
<h2 id="kprofefe-the-collector">Kprofefe: the collector</h2>
<p>The main responsability for the collector is to make the continuous profiling
magic to happen! It uses the same mechanism we already saw for the capture
kubectl plugin but it is a single binary and it can run as a cronjob.</p>
<pre><code>apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: kprofefe-allnamespaces
namespace: profefe
spec:
concurrencyPolicy: Replace
jobTemplate:
metadata:
spec:
template:
spec:
containers:
- args:
- --all-namespaces
- --profefe-hostport
- http://profefe-collector:10100
image: profefe/kprofefe:v0.0.8
imagePullPolicy: IfNotPresent
name: kprofefe
restartPolicy: Never
serviceAccount: kprofefe-all-namespaces
serviceAccountName: kprofefe-all-namespaces
schedule: '*/10 * * * *'
successfulJobsHistoryLimit: 3
</code></pre>
<p>You can run a single cronjob that will over all the pods across all the
namespaces or you can deploy multiple cronjobs, playing with the label selector
(-l) and the namespace selector (-n) you can configure the ownership for every
running cronjob. The reasons to split in multiple cronjobs can be:</p>
<ul>
<li>Scalability: one cronjob is not enough, so you can have one per namespace
for example</li>
<li>Time segmentation: if you have a single cronjob it means that all the pods
profiles will get captured with the same frequency, but you will may want to
get high frequent profiles for a specific subset of applications and less
dencity for others.</li>
</ul>
<p>Documentation about <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/">“Label and
Selector”</a>
for your reference.</p>
<p><em>Note: serviceAccount is required only if you have RBAC enabled (you should)
because the collector needs access to Kubernetes API to list/view pods across
all namespaces in this case.</em></p>
<h2 id="conclusion">Conclusion</h2>
<p>There is a lot to do in both the collector and kubectl plugins. I would like to
add logs and monitoring to the collector for example. The kubectl plugin get
profiles command needs some love, ideally using the same format that <code>kubectl
get</code> has via
<a href="https://github.com/kubernetes/cli-runtime/tree/master/pkg/printers">kubernetes/cli-runtime/pkg/printers</a>.
Try, contribute and <a href="https://twitter.com/gianarb">let me know</a>!</p>
Make boring tasks enjoyable with go and collyRecently I had the idea to update the conference page on my website with the end goal to make it a bit more structured. Where structured means a bit more reusable compared with the static HTML table I used to have. I mixed a bit of hacky Go, colly for scraping and that's who I did ithttps://gianarb.it/img/go.png2020-01-23T09:08:27+00:002020-01-23T09:08:27+00:00https://gianarb.it/blog/make-boring-task-enjotable-with-go-colly<p>Recently I had the idea to update the <a href="/conferences.html">conference</a> page on my website with the end
goal to make it a bit more structured. Where structured means a bit more
reusable compared with the static HTML table I used to have.</p>
<p>In the beginning, I decided to do an HTML table every year listing all the
conferences as a single row. It worked but I think at this point I can do
something even cooler with a single page for every conference talk with YouTube
and slides embedded, the abstract and few links to deep dive on the topic.</p>
<p>Jekyll has a cool feature called
<a href="https://jekyllrb.com/docs/collections/">collections</a>: “Collections are a great
way to group related content like members of a team or talks at a conference.” I
decided to do a “my_talks” collection.</p>
<p>I first added the right configuration in the <code>_config.yaml</code> and I added my first
conference in 2020, DevOps Pro in Vilnius (see you there!!).</p>
<pre><code>collections:
my_talks:
output: true
</code></pre>
<p>I have created my first talk as a markdown file, just as I do for my posts:</p>
<pre><code>---
title: Continuous Profiling Go Application Running in Kubernetes
date: 2020-03-24
slide:
embedSlide:
video:
embedVideo:
eventName: DevOps Pro Europe
eventLink: https://devopspro.lt/
city: Vilnius, Lithuania
---
Microservices and Kubernetes help our architecture to scale and to be
independent at the price of running many more applications. Golang provides a
powerful profiling tool called pprof, it is useful to collect information from a
running binary for future investigation. The problem is that you are not always
there to take a profile when needed, sometimes you do not even know when you
need to one, that's how a continuous profiling strategy helps. Profefe is an
open-source project that collects and organizes profiles. Gianluca wrote a
project called kube-profefe to integrate Kubernetes with Profefe. Kube-profefe
contains a kubectl plugin to capture locally or on profefe profiles from running
pods in Kubernetes. It also provides an operator to discover and continuously
profile applications running inside Pods.
</code></pre>
<p>As you can see I decided to set a bunch of variables that I will hope to re-use
where I will do the “single page” for each talk.</p>
<p>That’s it. All done: 2020 looks awesome and I added a for loop in the conference
page to print out the row as before:</p>
<pre><code><div class="row">
<h3></h3>
<div class="col-md-12">
<table class="table table-hover" id="">
<thead>
<tr>
<th>Date</th>
<th>Event</th>
<th>Talk</th>
<th>Slide</th>
</tr>
</thead>
<tbody>
</tbody>
</table>
</div>
</div>
</code></pre>
<p>In order to make everything a bit more reusable and organized this piece of code
is what Jekyll call <a href="https://jekyllrb.com/docs/includes/">include</a>. The way I
use it inside the conference page looks like:</p>
<pre><code>{ assign talks2020 = site.my_talks | where:'date', "2020" }
{ include talks_per_year.html year="2020" talks=talks2020 }
</code></pre>
<p>Everything is working fine, and I am pretty happy but I have over 6 years of
talks to convert in this new format, it means over 50 conferences to convert one
by one in the new format made of files and YAML.</p>
<p class="text-center"><img src="https://media.giphy.com/media/KFz5cubdh5eskezQ6d/giphy.gif" alt="https://media.giphy.com/media/KFz5cubdh5eskezQ6d/giphy.gif" class="img-fluid" /></p>
<h2 id="scraping-is-my-superpower">Scraping is my superpower</h2>
<p>I am not a fan of scraping things around and I never did that before, but hey!
This solution looks less boring that me doing it manually. I deep dive looking
for scraping libraries in new languages (yes you always have to learn new
languages when doing a new side project), but at the end I discovered
<a href="https://github.com/gocolly/colly">colly</a>: “Elegant Scraper and Crawler
Framework for Golang”. I decided to be elegant and effective.</p>
<h2 id="a-bit-about-colly">A bit about Colly</h2>
<p>I have to say that it took me less than 2 hours to hack a script in Go using
Colly that converted all my tables year by year from HTML to files with the
format you saw above. I also added some sweet sugar like:</p>
<ul>
<li>Be able to convert YouTube links when detected to their embeddable version</li>
<li>I converted and standardized the end/start date for the talks because it
changed year by year (I am lazy and unconsistent! Don’t tell anybody)</li>
</ul>
<p>It was soo easy that I didn’t write any test… yep, that’s it. The file name is a
bit weird but at the end it works, so who cares!</p>
<pre><code>$ tree ./_my_talks/
./_my_talks/
├── 2013-09-12-what-is-vagrant.markdown
├── 2014-02-c'è-un-modulo-zf2-per-tutto!---there-is-a-module-for-all.markdown
├── 2014-03-zend-queue.markdown
├── 2014-05-getting-start-chromecast-developer.markdown
├── 2014-05-vagrant,-riutilizzo-dell'infrastruttura---vagrant,-reuse-architecture.markdown
├── 2014-10-sviluppo-di-api-rest-con-zf2-&-mongodb.markdown
├── 2014-10-time-series-database,php-&-influx-db.markdown
├── 2015-01-angularjs-advanced-startup.markdown
├── 2015-06-delorean-made-in-home---reaspberry,-gobot-and-mqtt.markdown
├── 2015-07-joomla-and-scalability-with-aws-beanstalk.markdown
├── 2015-09-penny-php-middleware-framework.markdown
├── 2015-10-angularjs-in-cloud.markdown
├── 2015-10-doctrine-orm-cache-layer---it-is-not-a-boomerang.markdown
├── 2015-11-wordpress-and-scalability-with-docker.markdown
├── 2016-02-slimmer---poc-born-after-a-revolt-instant-vs-jenkins.markdown
├── 2016-03-a-zf-story:-parallel-made-easy.markdown
├── 2016-04-listen-your-infrastructure-and-please-sleep.markdown
├── 2016-05-continuous-delivery-with-jenkins-in-the-real-world.markdown
├── 2016-06-aws-under-the-hood.markdown
├── 2016-06-listen-your-infrastructure-and-please-sleep.markdown
├── 2016-06-parallel-made-easy.markdown
├── 2016-07-docker-1.12-and-orchestration-built-in.markdown
</code></pre>
<p class="text-center"><img src="https://i.kym-cdn.com/photos/images/newsfeed/000/345/534/4a2.jpg" alt="https://i.kym-cdn.com/photos/images/newsfeed/000/345/534/4a2.jpg" class="img-fluid" /></p>
<p>Anyway, let’s get to some snippets!</p>
<pre><code>type Talk struct {
Title string `yaml:"title"`
Date time.Time `yaml:"date"`
Slide string `yaml:"slide"`
EmbedSlide string `yaml:"embedSlide"`
Video string `yaml:"video"`
EmbedVideo string `yaml:"embedVideo"`
EventName string `yaml:"eventName"`
EventLink string `yaml:"eventLink"`
City string `yaml:"city"`
Links map[string]string `yaml:"links"`
}
var dateLayout = "_2 Jan 2006"
var year = "2020"
var outputDir = "/tmp"
var errorsToCheck = map[string]string{}
</code></pre>
<p>Those are the variables and struct I set. The Talk represent every single talk,
the dataLayout converts the way the end/start date is written into a time.Time
object. <code>year</code> is a parameter that tells which table to scrape, <code>outputDir</code>
tells where to place the files. Those 3 variables can be changed with cli flags:</p>
<pre><code>flag.StringVar(&year, "year", "2020", "The year used to identify the table to parse")
flag.StringVar(&dateLayout, "date-layout", "_2 Jan 2006", "The golang format layour to parse the event date column")
flag.StringVar(&outputDir, "output-dir", "/tmp", "Where to place the generated files")
flag.Parse()
</code></pre>
<p><code>errorsToCheck</code> is an easy way to collect all the errors for every run. I
printed them in a file, if the errors were easy to fix with a code change I did
that, if they were easier to change modifying the current conference page I did
that.</p>
<pre><code>// Instantiate default collector
c := colly.NewCollector(
// Visit only domains: coursera.org, www.coursera.org
colly.AllowedDomains("gianarb.it", "www.gianarb.it"),
// Cache responses to prevent multiple download of pages
// even if the collector is restarted
colly.CacheDir("./gianarb_cache"),
)
talks := []Talk{}
c.OnHTML("table[id=\""+year+"\"] tbody", func(e *colly.HTMLElement) {
e.ForEach("tr", func(_ int, row *colly.HTMLElement) {
talk := Talk{}
// for each line "tr" do amazing things
talks = append(talks, talk)
})
})
// Before making a request print "Visiting ..."
c.OnRequest(func(r *colly.Request) {
log.Println("visiting", r.URL.String())
})
err := c.Visit("https://gianarb.it/conferences.html")
if err != nil {
println(err)
}
</code></pre>
<p>This is how easy colly is to run. You have to configure the collector and with
the function <code>OnHTML</code> you can look for whatever you need to scrape. In this case
I was looking for the table identified with the <code>id</code> equals the year got from
the CLI. For each TR element I was creating a new talk to append in a slice. The
<code>talk</code> has to to be populated with the actual values scraped cell by cell. It
means that ForEach row we need to look for each td (cell in html) and based on
its index we can identify the content. In my case it looks like this:</p>
<pre><code>c.OnHTML("table[id=\""+year+"\"] tbody", func(e *colly.HTMLElement) {
e.ForEach("tr", func(_ int, row *colly.HTMLElement) {
talk := Talk{}
row.ForEach("td", func(_ int, el *colly.HTMLElement) {
switch el.Index {
case 0:
// Date
case 1:
// Event Name and conference URL (task.EventLink)
case 3:
// Video and slides link
}
})
talks = append(talks, talk)
})
})
</code></pre>
<p>I can show you how I coded the case 3, the one that looks for Video or Slides,
takes its link and in case of a YouTube Video it also converts the link into an
embeddable one:</p>
<pre><code>links := map[string]string{}
el.ForEach("a", func(_ int, el *colly.HTMLElement) {
switch el.Text {
case "Video":
talk.Video = el.Attr("href")
if strings.Contains(talk.Video, "youtube.com") {
u, err := url.Parse(talk.Video)
if err == nil {
talk.EmbedVideo = "https://www.youtube.com/embed/" + u.Query().Get("v")
} else {
errorsToCheck[row.Text+"/youtube_video_without_id"] = el.Text
}
} else {
errorsToCheck[row.Text+"/no_youtube_video"] = el.Attr("href")
}
case "Slides":
talk.Slide = el.Attr("href")
default:
links[el.Text] = el.Attr("href")
}
talk.Links = links
})
</code></pre>
<p>This is how I made a boring task enjoyable! And now I have all the talks (minus
two that didn’t get converted but I will add manually) converted and ready to be
rendered as posts.</p>
<h2 id="conclusion">Conclusion</h2>
<p>This post should not start a useless war between static side generator,
Wordpress or whatever. If you follow me on
<a href="https://twitter.com/gianarb">Twitter</a> you know that I tweeted recently about
changing Jekyll with something else, mainly because I was thinking how to make a
better use of the contents I create. Digging deeper with Jekyll I discovered
that for now I don’t need more than that and changing tool will end up as
useless and probably not that fun exercise. I am sure all other tools like
Wordpress, Hugo, Gatsby have something similar.</p>
My experience with Krew to manage kubectl pluginsKubectl plugins are extremely useful to provide a set of friendly utilities to interact with kubernetes in your environment. Krew is a project that helps you managing the plugin lifecycle. I have to add profefe to it and this is what I learned.https://gianarb.it/img/kubernetes.png2020-01-16T09:08:27+00:002020-01-16T09:08:27+00:00https://gianarb.it/blog/my-experiene-with-krew-to-manage-kubectl-plugins<p>I wrote a good number of kubectl plugins so far, but there is a lot more I can
do with them. Every time I write a new one I discover something new and that is
why I am always excited to see what will happen with the next one.</p>
<p><a href="[https://gianarb.it/blog/unit-testing-kubernetes-client-in-go](https://gianarb.it/blog/unit-testing-kubernetes-client-in-go)">“Unit Testing Kubernetes Client in
Go”</a>
and <a href="[https://gianarb.it/blog/kubectl-flags-in-your-plugin](https://gianarb.it/blog/kubectl-flags-in-your-plugin)">“Kubectl flags in your kubectl
plugin”</a>
are two of the lessons learned along the way.</p>
<p>With
<a href="[https://github.com/profefe/kube-profefe](https://github.com/profefe/kube-profefe)">kubectl-profefe</a>
I decided to have a look at
<a href="[https://github.com/kubernetes-sigs/krew](https://github.com/kubernetes-sigs/krew)">krew</a>.
It is a package manager for kubectl plugins. It is a plugin itself with the end
goal to help you installing and managing the lifecycle for your plugin.</p>
<pre><code>$ kubectl krew install profefe
</code></pre>
<p>It gives you the ability, with a single command to install, update or delete the
kubectl-profefe cli command.</p>
<p>Twitter got pretty excited recently about
<a href="[https://github.com/ahmetb/kubectl-tree](https://github.com/ahmetb/kubectl-tree)">kubectl-tree</a>,
a plugin from
<a href="[https://twitter.com/ahmetb](https://twitter.com/ahmetb)">@ahmetb</a> an old
friend of mine and active Kubernetes contributor and maintainer for krew as
well. It helps you to visualize kubernetes resources as a tree to simplify the
comprehension of the hierarchy and the connection between resources.</p>
<p>Two other examples that I would like to mention are from @ahmetb too. Kubectl
plugins don’t need to be extremely complicated, but you always have to keep in
mind the mantra “usability first.” It doesn’t matter how many lines of code you
write: the end goal should be to develop something usable and well-integrated
with kubernetes! <code>kubectl ctx</code> and <code>kubectl ns</code> are fabulous examples of
something easy but helpful. We switch between context and namespace more than
once a day between production clusters, local development, and so on. It is not
a very complicated thing to do natively: for example, changing context with
kubectl is just a matter of typing:</p>
<pre><code>$ kubectl config use new-context
</code></pre>
<p>Worst case scenario for the namespace, you have to type the <code>-n</code> flags every
time you run a kubectl command that is not in the namespace you have set by
default for the context you are using.</p>
<p>But <code>kubectl ctx</code> and <code>kubectl ns</code> simplify this process even more. You only
have to type:</p>
<pre><code>$ kubectl ctx new-context
</code></pre>
<p>Or</p>
<pre><code>$ kubectl ns new-namespace
</code></pre>
<p>If you are developing an open-source kubectl plugin and you need a friendly and
easy way to distribute it, you should have a look at krew. The publication
process is straightforward, <a href="[https://github.com/kubernetes-sigs/krew-index/pull/415](https://github.com/kubernetes-sigs/krew-index/pull/415)">this is the
PR</a>
I had to submit for profefe, you have to type some YAML as usual.</p>
Unit test kubernetes client in GoA flexible an easy to use testing framework makes all the difference. Kubernetes provides a fake client in Go that works like a charm.https://gianarb.it/img/kubernetes.png2020-01-10T09:08:27+00:002020-01-10T09:08:27+00:00https://gianarb.it/blog/unit-testing-kubernetes-client-in-go<p>I write a lot of operations and integrations with Kubernetes those days. You can
follow my journey in its dedicated section on this blog <a href="/planet/assemble-kubernetes.html">“Building
Kubernetes”</a>.</p>
<p>I had to write a function recently capable of filtering pods based on assigned
annotations.</p>
<pre><code class="language-go">
const (
ProfefeEnabledAnnotation = "profefe.com/enable"
)
// GetSelectedPods returns all the pods with the profefe annotation enabled
// filtered by the selected labels
func GetSelectedPods(clientset kubernetes.Interface,
namespace string,
listOpt metav1.ListOptions) ([]v1.Pod, error) {
target := []v1.Pod{}
pods, err := clientset.CoreV1().Pods(namespace).List(listOpt)
if err != nil {
return target, err
}
for _, pod := range pods.Items {
enabled, ok := pod.Annotations[ProfefeEnabledAnnotation]
if ok && enabled == "true" && pod.Status.Phase == v1.PodRunning {
target = append(target, pod)
}
}
return target, nil
}
</code></pre>
<p>This function is pretty easy, but it has a good amount of assertions that we can
check. Even more when we have a so well scoped functions writing tests should be
almost mandatory.</p>
<ul>
<li>The returned list of pods should only contains pods with the
<code>ProfefeEnabledAnnotation</code> set</li>
<li>The returned list of pods should only returns pods from the specified
<code>namespace</code></li>
<li>The returned list of pods should observe the filtering and label selection
criteria specified by <code>metav1.ListOptions</code></li>
</ul>
<p>Covering those use cases will give us a solid foundation to avoid regression
when this function will get more complicated (usually that’s the evolution for
successful piece of code).</p>
<h2 id="kubernetes-client-mock">Kubernetes Client Mock</h2>
<p>Kubernetes offers a simple and powerful <code>fake</code> client that has a very efficient
mechanism to simulate the desired output from a specific request, in our case
<code>clientset.CoreV1().Pods(namespace).List(listOpt)</code>. You have to pass the slice
of <code>runtime.Object</code> you desire when you create a new fake client. Awesome and
easy.</p>
<pre><code class="language-go">clientset: fake.NewSimpleClientset(&v1.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: "influxdb-v2",
Namespace: "default",
Annotations: map[string]string{},
},
}, &v1.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: "chronograf",
Namespace: "default",
Annotations: map[string]string{},
},
}),
</code></pre>
<p>For example this <code>clientset</code> will return two pods, one called <code>influxdb-v2</code> and
one called <code>chronograf</code>, but you can return what ever you need: Services,
Deployments, Ingress, Custom Resource Definition or even a mix of everything.</p>
<h2 id="in-practice">In practice</h2>
<p>I wrote a bunch of tests for
<a href="https://github.com/profefe/kube-profefe/blob/master/pkg/kubeutil/kube_test.go">kube-profefe</a>
that are using a fake client. You can get inspiration over there.</p>
<h2 id="conclusion">Conclusion</h2>
<p><code>fake</code> client is easy to use, so easy that since I added it in my tool chain for
some functions like the one I described here I efficiently do <code>TDD</code> because it
makes the iteration over my code way faster.</p>
Continuous profiling in Go with ProfefeTaking a snapshot at the right time is nearly impossible. A very easy way to fix this issue is to have a continuous profiling infrastructure that gives you enough confidence of having a profile at the time you need it.https://gianarb.it/img/profefe.png2020-01-03T09:08:27+00:002020-01-03T09:08:27+00:00https://gianarb.it/blog/go-continuous-profiling-profefe<p>There are a lot of articles about profiling in Go. Julia Evans for examples
wrote <a href="https://jvns.ca/blog/2017/09/24/profiling-go-with-pprof/">“Profiling Go programs with
pprof”</a> and I rely on
it when I do not remember how to properly use pprof.</p>
<p>Rakyll wrote <a href="https://rakyll.org/custom-profiles/">“Custom pprof profiles”</a>.</p>
<p><code>pprof</code> is a powerful tool provided by Go that helps any developer to figure out
what is going in the Go runtime. When you see a spike in memory in your running
container the next question is who is using all that memory. Profiles tell you
the answer.</p>
<p>But they need to be grabbed at the right time. The unique way to have a profile when
you need it is by taking them continuously. Based on your application you should
be able to specify how often you have to gather a profile.</p>
<p>This requires a proper infrastructure that we can call “Continuous profiles
infrastructure”. It is made of collectors, repositories and you need an API to
store, retrieve and query those profiles.</p>
<p>When we had to set it up at InfluxData we started to craft our own one until I
saw <a href="https://github.com/profefe/profefe"><code>profefe</code></a> on GitHub. What I love about
the project is its clear scope. It is a repository for profiles. You can push
them in Profefe and it provides an API to get them out, it servers the profiles in a
way that make them easy to visualize directly with <code>go tool pprof</code>, you can even
merge them together and so on. It also have a clear interface that helps you to
implement your own storage.</p>
<p>The project
<a href="https://github.com/profefe/profefe/blob/master/README.md">README.md</a> well
explains how it works but I am going to summarize the most important actions in
this article.</p>
<h2 id="getting-started">Getting Started</h2>
<p>There is a docker image that you can run with the command:</p>
<pre><code>docker run -d -p 10100:10100 profefe/profefe
</code></pre>
<p>You can push a profile in profefe:</p>
<pre><code>$ curl -X POST \
"http://localhost:10100/api/0/profiles?service=apid&type=cpu" \
--data-binary @pprof.profefe.samples.cpu.001.pb.gz
{"code":200,"body":{"id":"bo51acqs8snb9srq3p10","type":"cpu","service":"apid","created_at":"2019-12-30T15:18:11.361815452Z"}}
</code></pre>
<p>You can retrieve it directly via its ID:</p>
<pre><code>$ go tool pprof http://localhost:10100/api/0/profiles/bo51acqs8snb9srq3p10
Fetching profile over HTTP from http://localhost:10100/api/0/profiles/bo51acqs8snb9srq3p10
Saved profile in /home/gianarb/pprof/pprof.profefe.samples.cpu.002.pb.gz
File: profefe
Type: cpu
Time: Dec 23, 2019 at 4:06pm (CET)
Duration: 30s, Total samples = 0
</code></pre>
<p>There is a lot more you can do, when pushing a profile you can set key value
pairs called <code>labels</code> and they can be used to query a portion of the profiles.</p>
<p>You can use <code>env=prod|test|dev</code> or <code>region=us|eu</code> and so on.</p>
<p>Retrieving a profile only via ID it’s not the unique way to visualize it.
Profefe merges together profiles from the same type in a specific time range:</p>
<pre><code>GET /api/0/profiles/merge?service=<service>&type=<type>&from=<created_from>&to=<created_to>&labels=<key=value,key=value>
</code></pre>
<p>It returns the raw compressed binary, it is compatible with <code>go tool pprof</code> as
well as the single profile by id.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I didn’t develop profefe, <a href="https://github.com/narqo">Vladimir (@narqo)</a> is the
maintainer, I like it and how it is coded. I think it solves a very common
issue. He wrote a detailed post about his project
<a href="https://medium.com/@tvii/continuous-profiling-and-go-6c0ab4d2504b">“Continuous Profiling and Go”</a></p>
<blockquote>
<p>Wouldn’t it be great if we could go back in time to the point when the issue
happened in production and collect all runtime profiles. Unfortunately, to my
knowledge, we can’t do that.</p>
</blockquote>
<p>One of my colleague Chris Goller wrote a self contained AWS S3 implementation
that is now submitted as PR. We are running it since a couple of weeks now. It
is hard to onboard developers in a new tool, even more during Christmas but the
API layers makes it very comfortable and friendly to use. Next article will be
about what we did to get it running in Kubernetes continuously profiling our
containers.</p>
Year in reviewSummary about 2019, a year in review. There is not much more to say other than that! Happy new year!https://gianarb.it/img/me.jpg2019-12-30T06:08:27+00:002019-12-30T06:08:27+00:00https://gianarb.it/blog/year-in-review<p>I can’t say 2019 was great from a working point of view. I struggled a lot and I
probably had a small and dirty burnout. I learned about myself along the
way. A few folks that joined me at Influx left, I didn’t really enjoyed that
time and the massive growth the company had didn’t help me. I had some
difficulties finding my place and since there I didn’t really find the right
motivation to be productive such the passion for this job forces me to be.</p>
<p>Luckily I am surrounded by friends and colleagues with an open mind, I just had
to ask and to speak loud about my feelings, I always got back useful
conversations. I am sure every situation is different but other people
experienced similar situation and it is great to have them around. I think it is
getting better, I work in a different team now, the amount of YAML I have
to write decreases, I am back writing bugs and fixing them.</p>
<h2 id="open-source-is-about-collaboration">Open Source is about collaboration</h2>
<p>Open Source is part of my daily job and the reasons about why are well explain in
my first podcast ever! You can find it on <a href="https://www.stitcher.com/podcast/the-new-stack-makers/e/60409328?autoplay=true">The New
Stack</a>.</p>
<p>TLDR: I learned how to write code pinging people on IRC since day one. Different
communities helped me to improve in my daily job as nobody ever did. That’s why
open source is part of myself and I can’t stay without it. Even more now that I
have something to get back.</p>
<p>This year I stopped to do small projects in my GitHub profile all alone. It was
a natural decision that I didn’t took on purpose. Even more where I discovered
that my shitty useless code is gonna destroy the Arctic because <a href="https://www.youtube.com/watch?v=fzI9FNjXQ0o">GitHub spams it
there</a>.</p>
<p>I had the opportunity to discover a community called
<a href="https://github.com/testcontainers">testcontainers</a> they do cool things but you
know about them because I wrote <a href="/blog/testcontainers-go">“testcontainer library to programmatically
provision integration tests in Go with containers”</a>, I
tweet a lot about it and I spoke at DockerCon about the same topic <a href="https://www.youtube.com/watch?v=RoKlADdiLmU">“Write
Maintainable Integration Tests with
Docker”</a>.</p>
<p>Recently at Influx we was looking for a way to setup a continuous profiling
infrastructure, some work is still ongoing but Vladimir wrote a nice open source
project called <a href="https://github.com/profefe/profefe">profefe</a>, we deployed it and
I wrote a Kubernetes integration called
<a href="https://github.com/profefe/kube-profefe">kube-profefe</a>. It is now part of the
profefe organization and I am planning to write a series of posts about it, so
stay toned!</p>
<p>Join ongoing community and projects that you LIKE and USE is way better than
writing something alone that looks probably similar to something that already
exists. It is not easy, you have read more code, you have to reach out to other
people that will may be busy but I will keep doing it!</p>
<h2 id="meetups--conferences">Meetups & Conferences</h2>
<p>I run the CNCF Meetup in Turin, let me know if you would like to speak! I do it
because I work from home for a company based in San Francisco. They are far away
and I spend a lot of my working our by myself. My local community helps me to
develop great connections with people close to me! Ordering pizza, finding
locations, speakers, sponsors are unused tasks that I enjoy. All the videos are
available on <a href="https://www.youtube.com/channel/UCke-1vle73H9Dy4ojdfLw5A">YouTube</a>
some of them are in English other are not.</p>
<p>This year the plan is to run a nomad meetup, we will move office by office in
order to meet more people and to be cool companies or startups.</p>
<p>Would you like to sponsor, speak, host us?! Reach out
<a href="mailto:ciao@gianarb.it">ciao@gianarb.it</a> (Turin only locations).</p>
<p>I made way too many talks during the first part of the year (counting 11), and the
difficulties I had at work convinced me to take a break. I didn’t took any
flight since June (almost), I feel recharged now but I will keep a low
number of events this year. I would like to write more and to do more podcast.
Do you host one? Let me know!</p>
<h2 id="write">Write</h2>
<p>I wrote 26 articles. I am impressed by the number now that I see it. My articles
come from what I built I need to keep doing fun projects in order to have
something useful to write. I will probably stay focused on Extending Kubernetes
because I like how dynamic the code is, I would like to keep experimenting with
<a href="/blog/reactive-planning-and-reconciliation-in-go">reconciliations loops and reactive
planning</a> and to study Control
Theory because <a href="/blog/control-theory-is-dope">“it is done”</a>.</p>
<p><a href="https://www.cherryservers.com/?utm_source=garb&utm_medium=ftr&utm_campaign=drs">CherryServers</a>
is a cloud provider that I met at ContainerDays in Hamburg and since there we
loved each other! I can ping them and have fun on their platform as much as I
like and this is great. They do not have a Kubernetes story yet, let’s see if we
can do something about it! If you need to write an operator or a CSI plugin
(persistent storage), or who knows even a Cluster API implementation let me
know!</p>
<p>My first collaboration with a publisher was good but not excellent. I wrote a
report for O’Reilly called <a href="https://get.oreilly.com/ind_extending-kubernetes.html">“Extending Kubernetes”</a>. I didn’t get any
information from there about how it is going but it is a normal practice for “a
report” for what they told me.
I defined it “not excellent” because it does not look like a collaboration, it
is a one shot effort. I am happy to see it live because I like to write, but
it is not my best skill. This collaboration helped me to raise the bar.</p>
<p>In 2019 as I did for open source I would like to collaborate with other people,
maybe to write another book. Something is moving but let me know if you
have any idea.</p>
<h2 id="2020">2020!</h2>
<p>If I have to pick a word to describe 2019 I will use <code>join</code>. I <code>joined</code> a lot of
great people/teams embracing what they care or they were working on. I loved
that. I hope to keep doing it with the help of the communities I am part of like
Docker, observability, Kubernetes, CNCF, testcontainers. I hope to join more
people that shares my passions in order to improve and build something together.</p>
<p>In order to do that I need you all around! Reach out
<a href="https://twitter.com/gianarb">@gianarb</a>.</p>
<h2 id="home-sweet-home">Home sweet home!</h2>
<p>We have a new project! The most important one! We bought a house and there is a
lot of work to do! Look at how this wall is going down! Made in YAML.</p>
<div class="row">
<div class="col-md-6 offset-md-3">
<video class="embed-responsive" controls="">
<source class="embed-responsive-item" src="/img/destroy-home.mp4" type="video/mp4" />
</video>
</div>
</div>
Free PDFs about Docker from a CaptainThis article contains the ebooks listed in scaledocker.com. That link won't be available forever and I decided to move it here. If you are a beginner and you are happy to read about docker I got you covered. Security? There are 22 pages available for you as well. Enjoy.https://gianarb.it/img/the-fundamentals.jpg2019-12-18T06:08:27+00:002019-12-18T06:08:27+00:00https://gianarb.it/blog/scaledocker<p><img src="/img/mainbg_scaledocker.jpg" alt="" class="img-fluid" /></p>
<p>In the process of make some order in all my repositories, side projects and so
on my terminal crossed over a project I made in 2017 called scaledocker. The
idea was a full book. It turned out to be a 2 pdf one describing Docker for a
totally beginner, the second one about security.</p>
<p>Looking back I see why I am so happy to build and share what I do, I am still
like that today!</p>
<p>I was discussing with Jenny about what do with this project and we thought about
the possibility to refresh those contents as part of a series in docker.com.</p>
<p>I had a look around at this scenario, the contents are still good but I can’t
find the raw version of the PDF. It makes this work very hard. That’s why I
decided to archive it here.</p>
<p class="lead">scaledocker.com will stay up and running redirecting everybody to this article,
where you can find both PDFs. It is for sale, just <a href="mailto:ciao@gianarb.it">make an
offer</a> you can have it starting from Sept 2020.</p>
<p>This is not at end at all! There is an all section in my blog related to
<a href="/planet/docker.html">docker</a> and who knows what I will work on in 2020! I just
do not like the idea to have this project keeping a domain busy, I bet there are
people that can make a better use of it as I tried to do few years ago!</p>
<p>In order to stay in touch with me my website is a good way and I rumble a lot on
<a href="https://twitter.com/gianarb">twitter</a> as well!</p>
<h2 id="docker-the-fundamental">Docker the Fundamental</h2>
<p class="small">25 pages</p>
<p><img src="/img/the-fundamental-cover.jpg" alt="" class="img-fluid w-25 p-3 float-right" /></p>
<p>Getting Started about Docker, it covers basic concepts about what container
means and it’s a starting point to understand containers, Docker and all the
ecosystem.</p>
<ol>
<li>Introduction</li>
<li>Install Docker on Ubuntu 16.04</li>
<li>Install Docker on Mac</li>
<li>Install Docker on Windows</li>
<li>Run your first HTTP application</li>
<li>Docker engine architect</li>
<li>Image and Registry</li>
<li>Docker Command Line Tool</li>
<li>Volumes and File Systems 20</li>
<li>Network and Links</li>
<li>Conclusion</li>
</ol>
<p><a href="/downloads/the-fundamental.pdf" target="_blank">Open the pdf</a></p>
<h2 id="docker-security---play-safe">Docker Security - Play Safe</h2>
<p class="small">55 pages</p>
<p><img src="/img/container-security.png" alt="" class="img-fluid w-25 p-3 float-right" /></p>
<p>When you think about production everything stops to be a joke. Containers, cloud
computing, scalability is something that fit well with security? I have my
personal response for this question and with this paper I am going to show what
I mean with security and how containers, docker, Linux can make this real.</p>
<ol>
<li>Introduction</li>
<li>Mutual TLS and Security by default</li>
<li>Content Trust</li>
<li>Overlay Network</li>
<li>Docker Bench Security</li>
<li>Process Restriction and Capabilities</li>
<li>Open Source</li>
<li>Linux Kernel Security</li>
<li>Cilium</li>
<li>About your images</li>
<li>Secret Manager</li>
<li>Immutability</li>
</ol>
<p><a href="/downloads/play-safe.pdf" target="_blank">Open the pdf</a></p>
<h2 id="lets-move-on">Let’s move on!</h2>
<p>I hope this project will keep helping new folks to get onboard with Docker or to
figure out what they can do to improve security.</p>
<p>From what I can tell more than 2200 people required those PDFs! I am happy and
impressed. I can’t wait to see what we will do next!</p>
<p class="font-weight-bold">A big thanks to
<a href="https://www.cherryservers.com/?utm_source=garb&utm_medium=ftr&utm_campaign=drs">CherryServer</a>
for hosting the reverse proxy from scaledocker.com.</p>
Programmatically Kubernetes port forward in GoDepending on your networking configuration port forwarding will may be the unique way for you to reach pods or services running in Kubernetes. When you develop a CLI integration that has to interact with pods running inside the cluster you can programmatically do a port forwarding in golang.https://gianarb.it/img/gianarb.png2019-12-05T06:08:27+00:002019-12-05T06:08:27+00:00https://gianarb.it/blog/programmatically-kube-port-forward-in-go<p>Along the way I saw at least two different ways to manage Kubernetes clusters
from a networking prospective. Some companies configure a VPN inside the
Kubernetes Cluster, in this way a developer connected in the VPN can reach pods
and services.</p>
<p>It is not mandatory but suggested having a good network segmentation in order
to be able to manage what a person connected in the VPN can touch and see.
Achieving this level of control is not easy in Kubernetes a lot of the open
source CNI plugin does not have this feature at all and I understand why in
operations this is evaluated as a safe approach. It is very convenient if close
an eye because pods and services are just IPs that you can reach from your
laptop and if you configure the VPN to push the Kubernetes DNS you can also
resolve them as DNS lookup.</p>
<p>The alternative I saw is to lock everybody out leaves as unique way to interact
with a service or a pod the command <code>kubectl port-forward</code>. In this way the
authentication and authorization method in Kubernetes allows you to decide who
can do port-forwarding on what based on namespace for example. Or at least you
can use Kubernetes Audit logs to figure out who did port forwarding if something
bad happens.</p>
<p>We tried both ways, I was to one pushing for the first, but we never achieved a
good segmentation and at some point I got locked down, sadly as it sound. Anyway
I like to automate things and I had to figure out a way to make my scripts to
work with this new approach.</p>
<p><img src="/img/sub.jpg" alt="" class="img-fluid" /></p>
<p>I started to dig in the <code>kubectl</code> code because we all know that it is capable of
doing the port forwarding. I had some trouble figuring out the right parameters
and to make them to work but at the end I did it! So here we are! If I can do it
you can do it as well!</p>
<p>The main repository with the code and an example is in
<a href="https://github.com/gianarb/kube-port-forward">github.com/gianarb/kube-port-forward</a>,
you can run it there. I am gonna explain it a bit here.</p>
<p>It is a simple CLI that mocks what <code>kubectl port-forward</code> already does but I
extrapolated the code needed to do and control a port forwarding. I will write
here as soon as the reason about why I did that is open source, I am telling it
to you right now STAY TUNED! It will be great!</p>
<p>First of all I used the <code>k8s.io/cli-runtime/pkg/genericclioptions</code> library to
configure a stream, we already used in the <a href="/blog/kubectl-flags-in-your-plugin">blog post about writing a CLI that
uses the same flags as the kubectl</a>. A
stream is a <code>struct</code> used by different <code>kubernetes</code> service when they need to
get or print information from a stream, in this case I am using <code>os.Stdout</code>,
<code>os.Stdin</code>, <code>os.Stderr</code> for simplicity, but where I do not need to print out the
output I use a <code>bytes.Stream</code> like this:</p>
<pre><code class="language-go">var berr, bout bytes.Buffer
buffErr := bufio.NewWriter(&berr)
buffOut := bufio.NewWriter(&bout)
</code></pre>
<p>In order to make this code easy to read I had a structure to request the port
forwarding for a pod:</p>
<pre><code class="language-go">type PortForwardAPodRequest struct {
// RestConfig is the kubernetes config
RestConfig *rest.Config
// Pod is the selected pod for this port forwarding
Pod v1.Pod
// LocalPort is the local port that will be selected to expose the PodPort
LocalPort int
// PodPort is the target port for the pod
PodPort int
// Steams configures where to write or read input from
Streams genericclioptions.IOStreams
// StopCh is the channel used to manage the port forward lifecycle
StopCh <-chan struct{}
// ReadyCh communicates when the tunnel is ready to receive traffic
ReadyCh chan struct{}
}
</code></pre>
<p>And I wrote the function that actually does the port forward:</p>
<pre><code class="language-go">func PortForwardAPod(req PortForwardAPodRequest) error {
path := fmt.Sprintf("/api/v1/namespaces/%s/pods/%s/portforward",
req.Pod.Namespace, req.Pod.Name)
hostIP := strings.TrimLeft(req.RestConfig.Host, "htps:/")
transport, upgrader, err := spdy.RoundTripperFor(req.RestConfig)
if err != nil {
return err
}
dialer := spdy.NewDialer(upgrader, &http.Client{Transport: transport}, http.MethodPost, &url.URL{Scheme: "https", Path: path, Host: hostIP})
fw, err := portforward.New(dialer, []string{fmt.Sprintf("%d:%d", req.LocalPort, req.PodPort)}, req.StopCh, req.ReadyCh, req.Streams.Out, req.Streams.ErrOut)
if err != nil {
return err
}
return fw.ForwardPorts()
}
</code></pre>
<p>An exercise that I can leave for you is to add Service support to this function,
you can open a PR if you like on
<a href="https://github.com/gianarb/kube-port-forward">github.com/gianarb/kube-port-forward</a>.</p>
<p>The <code>Stop</code> and <code>Ready</code> channels are crucial to manage the port forward because
as you see in the example it is a blocking operation it means that it will
luckily always run inside a goroutine. Those two channels gives you what you
need to understand when the port forward is ready to get traffic <code>ReadyCh</code> and
you have the capabilities to stop it <code>StopCh</code>.</p>
<p>My example is basic, I am closing the port forwarding when the <code>SIGTERM</code> signal
gets notified:</p>
<pre><code class="language-go">sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigs
fmt.Println("Bye...")
close(stopCh)
wg.Done()
}()
</code></pre>
<p>I just wait until the readyCh tells me that the connection is
up and running</p>
<pre><code class="language-go">select {
case <-readyCh:
break
}
println("Port forwarding is ready to get traffic. have fun!")
</code></pre>
<p>As soon as I coded this feature I saw that it was gonna be an easy but useful
post. I wrote a <a href="/blog/extending-kubernetes-oreilly">report with O’Reilly</a> about
how to extend Kubernetes, you can find more about Go and Kube there. It is a
free PDF.</p>
<p>I hope you enjoyed it and <a href="https://twitter.com/gianarb">let me know</a> what cool
things you are gonna do port-forwarding the universe!</p>
OpenTelemetry the instrumentation library, I hopeOpenTelemetry, OpenCensus, OpenTracing, Open your hearthttps://gianarb.it/img/gianarb.png2019-11-20T08:08:27+00:002019-11-20T08:08:27+00:00https://gianarb.it/blog/opentelemetry-the-instrumentation-library-i-hope<p>Hello! If you follow my rumbling here or on
<a href="https://twitter.com/gianarb">twitter</a> you know that I like to speak about
observability and tracing.</p>
<p>If you don’t know what I am speaking about this is an
<a href="/blog/faq-distributed-tracing">FAQ</a> about distributed tracing and something
about <a href="/blog/what-is-distributed-tracing-opentracing-opencensus">OpenTracing and OpenCensus</a>.</p>
<p>Observability is the ability to figure out what is going on in your application
from the outside. In order to do that you need to instrument your applications
in order to expose the right information.</p>
<p>The instrumentation is not easy, there are too many developers and too many
opinions, too many languages but in order observe a system that cross all the
applications and services everything needs to come together in the same way.
Otherwise the aggregation will become very a complicated job.</p>
<p>When you instrument an application there is a lot of code to write and inject,
you can not do or change it based on the vendor or services you are using to
store your telemetry: Zipkin, InfluxDB, NewRelic, HoneyComb and so on.</p>
<p>That’s why over the last couple of years big foundations and companies such as
LightSteps, Google, CNCF, Uber tried to get their hands on the democratization
of code instrumentation. First with OpenTracing, after that with OpenCensus and
now with OpenTelemetry that is the merge between OpenCensus and OpenTracing.</p>
<p>At the beginning when this project went out a was very tired and stressed about
the topic. I made a workshop last year about code instrumentation at <a href="https://cloudconf.it">the
CloudConf</a> and I wish it was easier to prepare and
develop. At the end attendees were satisfied btw, so I am happy enough.</p>
<p>Since the beginning I had a very bad feeling about OpenTracing and OpenCensus,
it is necessary as a project but the fact that we had two ways because they
didn’t want to agree on only one for me was unbelievable.</p>
<p>Anyway, now that I pushed that feeling back I will give it another try. I will
get my <a href="https://github.com/gianarb/workshop-observability">observability
workshop</a> and I will refresh
it to use OpenTelemetry because as I said, we need a way to instrument
applications, cross vendor and cross languages.</p>
<p>Here some link about it:</p>
<ul>
<li><a href="https://opentelemetry.io/">opentelemetry.io</a></li>
<li><a href="https://github.com/open-telemetry">github.com/open-telemetry</a></li>
<li><a href="https://lists.cncf.io/g/cncf-opentelemetry-community">Mailing List</a></li>
</ul>
<p>At KubeCon 2019 <a href="https://twitter.com/lizthegrey">lizthegrey</a> gave a demo about
OpenTelemetry and I am confident that my experience will be a bit better.</p>
<p>It is not easy to democratize something, even less when you need to change the
habit for developers across programming languages. But that’s the goal for
OpenTelemetry and I think we need to get there and to make it a commodity. It is
not a joke!</p>
<p>If you would like to help me, let me know!</p>
o11y.guru introduction and first set of iterationsPart of the o11y.guru series this post is an introduction for this side project and it describes the first architecture designed for the website.https://gianarb.it/img/gianarb.png2019-11-09T08:08:27+00:002019-11-09T08:08:27+00:00https://gianarb.it/blog/o11yguru-introduction<div class="jumbotron">
<h2 id="introduction">Introduction</h2>
<p>This article is part of a series I am writing about a
side project I made called <a href="https://o11y.guru">o11y.guru</a>. Who knows what this
series or the project itself will become. The reason why I started it was to
have my <strong>wonderland</strong>. A place where I was able to do my mistakes without any
intermediary.</p>
<p>This series is about my journey there. I will keep the this <code>Introduction</code>
common to all the articles part of this series and I will keep a <code>Table of
Content</code> up to date. The best way to follow my journey here is to subscribe to
the RSS feed or to follow me on <a href="https://twitter.com/gianarb">@twitter</a>.</p>
<h3 id="what-is-this-project">what is this project?</h3>
<p>I had this idea to create a mechanism to enable other people that uses twitter
to follow a group of leaders in a particular space. I decided to start from the
observability (#o11y) because it is the area where I am in at the moment.</p>
<p><a href="https://o11y.guru">o11y.guru</a> is pretty simple, a list of faces and a
button, they you press it and if you will authorize the twitter application you
will get to follow them.</p>
<h3 id="table-of-content">Table of Content</h3>
<ol>
<li><a href="/blog/o11yguru-introduction">First day and first set of iterations</a></li>
<li>Build process and automation driven by simplicity</li>
<li>Monitoring and instrumentation with Honeycomb</li>
<li><a href="/blog/o11yguru-history-first-bug">The history of the first bug</a></li>
<li>OpenTelemetry it is time to embrace a unicorn standard</li>
<li>The magic of structured logging</li>
<li>Infrastructure monitoring with InfluxCloud</li>
<li>Infrastructure as code with Terraform and CherryServer. First deploy.</li>
<li>FAQ</li>
</ol>
</div>
<p><img src="/img/o11y-guru-series/index.png" alt="" class="img-fluid" /></p>
<p>olly.guru is a website that I wrote in Go. It lists a group of people active on
twitter that I like to follow around monitoring, reliability, observability. It
allows you to follow them all in once as well.</p>
<p>A few years ago when I was developing almost in PHP somebody from the community,
I don’t remember who was, I am getting old, did a similar website and I thought
it was a great idea.</p>
<p>Since then, the project was in the back, in my mind. I am a lazy person when it
comes to writing code. I think there is enough useless code around, and I don’t
want to incentive that practice. That’s why I tend to write as less code I can.</p>
<p>There are a bunch of reasons why I changed my mind, and I started to do it:</p>
<p>Better mood and I had to try Honeycomb.io, but I never had the right
opportunity. I didn’t want to try it with a demo running on my laptop. I
have a couple of new friends from CherryServer that supports my crazy ideas, and
I was looking for a reason to glue a bunch of reliable infrastructure as code that I
actually like. Even if it is usually a task I hate, mainly because there is no code
involved.</p>
<p>As Docker Captain and CNCF Ambassador, I have the feeling than a project like
that can be re-used.</p>
<p>I made the mistake that everyone does; I started to think about cool technologies
and not the problem I was going to solve or the project I was going to write.</p>
<p>I made a react application, just the foldering, and I quickly realized that I do not
know how to React, and I was wasting my time. But for this project, I got lucky
enough to keep going. I started to think about the problem again, and I decided
to make it as simple as possible. In practice almost everything gets generated
by an html template and a piloted by a list of names in a <code>txt</code> file. Very easy!</p>
<pre><code>.
├── cmd
│ ├── generate
│ └── www
├── Dockerfile
├── go.mod
├── go.sum
├── index.tmpl
├── Makefile
├── people
│ └── people.go
├── people.txt
├── style
│ ├── css
│ ├── fonts
│ ├── img
│ ├── index.html
│ ├── js
│ ├── node_modules
│ ├── package.json
│ ├── package-lock.json
│ └── scss
├── vendor
└── www
</code></pre>
<p>You can see the shape of the project; it has a minimal amount of technologies
involved: Go, HTML, Bootstrap 4, and a bit of Javascript. I started from the
<code>style</code> directory. I use it for prototyping the HTML, CSS part. I am far away to
be good with colors and CSS, and I am cool with that, we do not like each other.
So I do all my tests there, and when I am ready, I port the <code>style/index.html</code>
into <code>index.tmpl.</code> I started from a Bootstrap 4 layout already done as you can
see. It is in their documentation.</p>
<p><img src="/img/o11y-guru-series/sheldon.jpeg" alt="" /></p>
<p><code>index.tmpl</code> is the template I use to render the actual homepage. <code>www</code> is the
target destination for all the static files and the generated index page. I use
Make to copy files from <code>style</code> into <code>www,</code> and I wrote a CLI that generates the
HTML and it populates it with Twitter informations. It is inside <code>./cmd/generate.</code></p>
<p><code>./people.txt</code> is the list of twitter gurus. It is just a list:</p>
<pre><code>gianarb
rakyll
</code></pre>
<p>The <code>cmd/generate</code> reads that file and, it gets the information it needs from
the Twitter API like user bio, avatar and it renders the <code>./index.tmpl</code>
into the actual index inside the <code>www</code> folder.</p>
<p><code>./cmd/www</code> is an HTTP server written Go that serves the content of the <code>www</code>
directory. Plus it uses:</p>
<pre><code>github.com/dghubble/go-twitter
github.com/dghubble/gologin/v2
</code></pre>
<p>To manage the Twitter authentication flow.</p>
<p>I am sure you are wondering, “is he gonna open-source that!?”. I am. Not now.
The project needs to refactoring and some code needs to get stronger around
instrumentation and logging. As you see in the introduction, I am using this
experience as a use case to write down a bunch of practices I like or that I
would like to investigate.
So stay tuned! It will be available very soon.</p>
<h2 id="tldr-lesson-learned">tldr lesson learned</h2>
<p>Some of lessons I learned comes from how I am, but hey, this is my blog, I can
do whatever I like!
It is refreshing to start a project, but <strong>it is way cooler to have something to
show</strong>. So be careful when you start it, get it right so you won’t get tired.</p>
<p><strong>Set clear goals</strong>, and see point 1, they need to be very easy to
achieve, at least at the beginning.</p>
<p><strong>Do not type on your terminal, but write bash scripts.</strong> I started to do this
months ago at work. Bash scripts are way better than random commands in a
terminal because you can move them around composing way more powerful workflows.
You won’t lose them. That’s how I built my Makefile, just from the terminal
history or from the shortcut I made along the way.
Often a well-done <strong>dotenv file is enough to manage everything you need</strong>.</p>
<h2 id="lets-get-to-some-code">let’s get to some code</h2>
<p>I told you about bash scripts and Makefile, I will write a post about automation
for a small project, but this is part of my Makefile:</p>
<pre><code>style/build:
cd ./style && npm install
cp -r ./style/node_modules/jquery/dist/jquery.js ./style/js/jquery.js
cp ./style/node_modules/@fortawesome/fontawesome-free/js/all.js ./style/js/
cp -r ./style/node_modules/@fortawesome/fontawesome-free/webfonts ./style/fonts
cd ./style && npm run scss
style/start: style/build
cd ./style && npm start
style/compile: style/build
rm -rf ./www
mkdir ./www
cp -r ./style/img ./www
cp -r ./style/fonts ./www
cp -r ./style/css ./www
cp -r ./style/js ./www
</code></pre>
<p>People can do the same with npm and, a hundred node packages, I like to keep
things simple at this point to avoid unnecessary blockers that will get my
tired. This is how I manage the <code>style</code> directory and how I build the <code>www</code> one.</p>
<blockquote>
<p>With blockers I mean: googling around for things that should be easy.</p>
</blockquote>
<pre><code class="language-go">flag.StringVar(&flags.consumerKey, "consumer-key", "", "Twitter Consumer Key")
flag.StringVar(&flags.consumerSecret, "consumer-secret", "", "Twitter Consumer Secret")
flag.StringVar(&flags.accessToken, "access-token", "", "Twitter access key")
flag.StringVar(&flags.accessSecret, "access-secret", "", "Twitter access secret")
flag.StringVar(&flags.guruFile, "guru-file", "", "File that contains the guru's name")
flag.StringVar(&flags.indexTemplate, "index-template", "", "File that contains the guru's name")
flag.Parse()
flagutil.SetFlagsFromEnv(flag.CommandLine, "TWITTER")
config := oauth1.NewConfig(flags.consumerKey, flags.consumerSecret)
token := oauth1.NewToken(flags.accessToken, flags.accessSecret)
httpClient := config.Client(oauth1.NoContext, token)
// Twitter client
client := twitter.NewClient(httpClient)
// Verify Credentials
verifyParams := &twitter.AccountVerifyParams{
SkipStatus: twitter.Bool(true),
IncludeEmail: twitter.Bool(true),
}
_, _, err := client.Accounts.VerifyCredentials(verifyParams)
if err != nil {
println(err.Error())
os.Exit(1)
}
gurus := []*twitter.User{}
lines, err := people.ReadLineByLine(flags.guruFile)
if err != nil {
println(err.Error())
os.Exit(1)
}
for _, eachline := range lines {
user, _, err := client.Users.Show(&twitter.UserShowParams{
ScreenName: eachline,
})
</code></pre>
<p>The generate command is straightforward, I get over the <code>people.txt</code> file line
by line, and for every record, I get information about the user. When I have the
slice of gurus populated I render the template:</p>
<pre><code class="language-go">t, err := template.ParseFiles(flags.indexTemplate)
if err != nil {
panic(err)
}
err = t.Execute(os.Stdout, Render{
Gurus: gurus,
})
</code></pre>
<p>I decided to print the HTML into the stdout because it is way easier to use <code>></code>
other than accepting another parameter to specify the target output.</p>
<p>LDD: laziness driven development.</p>
<p>The <code>cmd/www</code> uses the same people.txt file to know who to follow when the user
presses the <code>Follow</code> button and authorize the twitter application:</p>
<pre><code class="language-go">for _, eachline := range lines {
if strings.EqualFold(eachline, me.ScreenName) {
continue
}
time.Sleep(5 * time.Second)
err = newFriendship(ctx, twitterClient, eachline)
if err != nil {
logger.Warn(err.Error(), zap.String("follower_screenname", eachline), zap.Error(err))
}
}
</code></pre>
<h2 id="the-project-in-the-project">The project in the project</h2>
<p>This series of posts I am writing is a side project in the side project. As I
wrote earlier I like to share what and why I do things. I hope to keep having
practical experiences to write down.</p>
<p>The high level expectation I set are:</p>
<ol>
<li>Have fun</li>
<li>Create a good network of followers on twitter that likes to speak about
observability</li>
<li>Learning how Honeycomb works and why everybody says that it sounds like magic</li>
<li>Writing down something about code instrumentation, infrastructure as a code and
automation</li>
<li>Exercise my experience as a decision maker driven by simplicity and
efficiency.</li>
<li>I hope to work with a couple of friends from Docker, HashiCorp,
CherryServier, InfluxData, HoneyComb to help me out with secret management,
monitoring, terraform and, automation in order to build the coolest project
ever. You will get an email from me (or reach out if you have suggestions).</li>
</ol>
<h2 id="thats-it">That’s it</h2>
<p>I am sure it gets out clearly from this article the friction between the
excitement about having an idea and the effort to make it real, even when it is
as simple as a single html page.. I struggle with
that all the time, and the laziness usually wins. Will this time be different?!
Well, I have a domain that is not a blank page. I think it is a good starting
point.</p>
<p>Time matter and have fun!</p>
<p><img src="/img/o11y-guru-series/sleep.jpg" alt="" class="img-fluid" /></p>
o11y.guru the history of the first bugPart of the o11y.guru series this is about the first bug I discovered with the help of honeycomb and how I had to fix it twice in order to make it to work.https://gianarb.it/img/gianarb.png2019-11-07T08:08:27+00:002019-11-07T08:08:27+00:00https://gianarb.it/blog/o11yguru-history-first-bug<div class="jumbotron">
<h2 id="introduction">Introduction</h2>
<p>This article is part of a series I am writing about a
side project I made called <a href="https://o11y.guru">o11y.guru</a>. Who knows what this
series or the project itself will become. The reason why I started it was to
have my <strong>wonderland</strong>. A place where I was able to do my mistakes without any
intermediary.</p>
<p>This series is about my journey there. I will keep the this <code>Introduction</code>
common to all the articles part of this series and I will keep a <code>Table of
Content</code> up to date. The best way to follow my journey here is to subscribe to
the RSS feed or to follow me on <a href="https://twitter.com/gianarb">@twitter</a>.</p>
<h3 id="what-is-this-project">what is this project?</h3>
<p>I had this idea to create a mechanism to enable other people that uses twitter
to follow a group of leaders in a particular space. I decided to start from the
observability (#o11y) because it is the area where I am in at the moment.</p>
<p><a href="https://o11y.guru">o11y.guru</a> is pretty simple, a list of faces and a
button, they you press it and if you will authorize the twitter application you
will get to follow them.</p>
<h3 id="table-of-content">Table of Content</h3>
<ol>
<li><a href="/blog/o11yguru-introduction">First day and first set of iterations</a></li>
<li>Build process and automation driven by simplicity</li>
<li>Monitoring and instrumentation with Honeycomb</li>
<li><a href="/blog/o11yguru-history-first-bug">The history of the first bug</a></li>
<li>OpenTelemetry it is time to embrace a unicorn standard</li>
<li>The magic of structured logging</li>
<li>Infrastructure monitoring with InfluxCloud</li>
<li>Infrastructure as code with Terraform and CherryServer. First deploy.</li>
<li>FAQ</li>
</ol>
</div>
<h2 id="the-history-of-the-first-bug">The history of the first bug</h2>
<p>After the first deploy I used my twitter account
<a href="https://twitter.com/dev_campy">@gianarb</a> and
<a href="https://twitter.com/dev_campy">@devcampy</a> to try the application. I have also
asked a friend to try it out.</p>
<p>So far so good, the way I coded the following workflow is very basic, and
probably it will quickly reach its scalability limit. It is a loop with a
<code>time.Sleep(5 * time.Second)</code> break between each account to avoid the Twitter
rate limit.</p>
<pre><code class="language-go">
for _, guru := range gurus {
time.Sleep(5 * time.Second)
err = newFriendship(ctx, twitterClient, guru)
if err != nil {
logger.Warn(err.Error(), zap.Error(err))
}
}
</code></pre>
<p>No retry or things like that for now. Very simple. I hope to iterate on it in
the future when it will start to not working well enough anymore.</p>
<p>It does not report any error if the Twitter API request to follow a person
fails, it just go to the next one. All three tests went well for what I was able
to say, all three accounts followed the gurus.</p>
<p>One of the first benefit about using HoneyComb is that out of the box they are
able to detect errors looking at the events you return and the graph is made by
them. Just clicking around to their UI I ended up with weirdness like this graph:</p>
<p><img src="/img/o11y-guru-series/first-bug-http-status.png" alt="Requests break down by HTTP Status" class="img-fluid" /></p>
<p>I noticed some <code>500</code> error page, and I do not like that. As you can see
there is an <code>Error</code> tab, built by Honeycomb again and this is what it showed to me:</p>
<p><img src="/img/o11y-guru-series/first-bug-span-with-error.png" alt="Span with an error" class="img-fluid" /></p>
<p>At this point it is clear to me where the problem is: “You can’t follow
yourself”. It sounds reasonable.</p>
<p>I changed the code and I added a simple <code>if</code> statement to skip the guru if it is
the person actually following all the other people.</p>
<pre><code class="language-go">// me comes from above when I validate that the token behaves to a user.
if guru == me.ScreenName {
continue
}
</code></pre>
<p>It does not sound trivial at all but when I tried the fix didn’t work.</p>
<p><img src="/img/o11y-guru-series/rambo.jpg" alt="" class="img-fluid" /></p>
<p>I decided to face up the problem differently. <em>Spoiler alert: I didn’t
write any unit test yet. Feel free to leave now.</em></p>
<p>Looking at the trace I knew I had set for every <strong>following request</strong> the guru name
to follow and at the root span I had who required to follow the
gurus. In practice, I had in the root span <code>required_by=me.ScreenName</code>, and for every
guru its span with their name. The next image has those two
spans side by side:</p>
<ul>
<li>At the left the span <code>newFriendship</code> describes a single following action (a
twitter create friendship api request). As you can see it has the <code>error="you
can't follow yourself"</code> and the <code>follower_screenname=gianarb</code>.</li>
<li>The one to the right is the root span, it has the <code>required_by=GianArb</code> field,
it is the <code>me.ScreenName</code> variable.</li>
</ul>
<p><img src="/img/o11y-guru-series/first-bug-compare-spans.png" alt="" class="img-fluid" /></p>
<p>Looking at this span the situation is clear, I was comparing <code>GianArb</code> the
<code>required_by</code> variable that you see in the right with <code>gianarb</code>, the
<code>follower_screenname</code> you see at the left span.</p>
<p>At the end of the story the check needs to be case-insensitive. And that’s how
it is now:</p>
<pre><code>if strings.EqualFold(guru, me.ScreenName) {
continue
}
</code></pre>
<p>This is the history of the first but I randomly discovered and I had to fix
twice for <code>o11y.guru</code>.</p>
O'Reilly Report Extending KubernetesI wrote a report with O'Reilly called: Extending Kubernetes.https://gianarb.it/img/hero/cat-sleep.jpeg2019-10-07T08:08:27+00:002019-10-07T08:08:27+00:00https://gianarb.it/blog/extending-kubernetes-oreilly<p>A few months ago I wrote a report with O’Reilly called <a href="https://get.oreilly.com/ind_extending-kubernetes.html">Extending
Kubernetes</a>. I realized I
didn’t share this major achievement here with all of you!</p>
<p>It comes from my experience not from ops but as a developer that works with
Kubernetes.</p>
<p>K8S requires a big effort in terms of maintenance and setup, who says something
different lies. Complexity is not too bad and sometime it is a requirement, a
good way to justify it is to use kubernetes as much as you can.</p>
<p>Operations and teams need a good UX, Kubernetes is extremely extensible with
custom resource definitions, kubectl plugins, controller, shared informers,
audit logs and operators.</p>
<p>This report is about it. I meanly use Go for the examples but a lot of them can
be re-written using any SDK provided by the Kubernetes community.</p>
<p>It is a practical report, with code examples and ideas about what you can do to
integrate your day to day operations with Kubernetes in order to share the pain
with the developers.</p>
<p><img src="/img/white-polar.jpeg" alt="Sleepy" /></p>
<p>Now that I recovered from the effort of writing it, stay tuned because I am
looking for something else to do!</p>
<p>Read it and let me know if you like it via
<a href="https://twitter.com/gianarb">@gianarb</a>.</p>
<p class="small">Hero image via <a href="https://pixabay.com/en/fractal-complexity-geometry-1758543/">Pixabay</a></p>
kubectl flags in your pluginDevelop cure custom kubectl plugins with friendly flags from the kubectlhttps://gianarb.it/img/gianarb.png2019-10-07T08:08:27+00:002019-10-07T08:08:27+00:00https://gianarb.it/blog/kubectl-flags-in-your-plugin<p>This is not at all a new topic, no hacking involved, but it is something
everybody needs to know where we design kubectl plugin.</p>
<p>I was recently working at one and I had to make the user experience as friendly
as possible compared with the <code>kubectl</code>, because that’s what a good developer does!
Tricks other developers to make their life comfortable, if you are used to do:</p>
<pre><code class="language-bash">$ kubectl get pod -n your-namespace -L app=http
</code></pre>
<p>To get pods from a particular namespace <code>your-namemespace</code> filtered by label
<code>app=http</code> and your plugin does something similar or it will benefit from an
interaction that remembers the classic <code>get</code> you should re-use those flags.</p>
<p>Example: design a <code>kubectl-plugin</code> capable of running a <code>pprof</code> profile against a
set of containers.</p>
<p>My expectation will be to do something like:</p>
<pre><code class="language-bash">$ kubectl pprof -n your-namespace -n pod-name-go-app
</code></pre>
<p>The Kubernetes community writes a lot of their code in Go, it means that there
are a lot of libraries that you can re-use.</p>
<p><a href="https://github.com/kubernetes/cli-runtime/tree/master/pkg/genericclioptions">kubernetes/cli-runtime</a>
is a library that provides utilities to create kubectl plugins. One of their
packages is called <code>genericclioptions</code> and as you can get from its name the goal
is obvious.</p>
<pre><code class="language-go">
// import "github.com/spf13/cobra"
// import "github.com/spf13/pflag"
// import "k8s.io/cli-runtime/pkg/genericclioptions"
// Create the set of flags for your kubect-plugin
flags := pflag.NewFlagSet("kubectl-plugin", pflag.ExitOnError)
pflag.CommandLine = flags
// Configure the genericclioptions
streams := genericclioptions.IOStreams{
In: os.Stdin,
Out: os.Stdout,
ErrOut: os.Stderr,
}
// This set of flags is the one used for the kubectl configuration such as:
// cache-dir, cluster-name, namespace, kube-config, insecure, timeout, impersonate,
// ca-file and so on
kubeConfigFlags := genericclioptions.NewConfigFlags(false)
// It is a set of flags related to a specific resource such as: label selector
(-L), --all-namespaces, --schema and so on.
kubeResouceBuilderFlags := genericclioptions.NewResourceBuilderFlags()
var rootCmd = &cobra.Command{
Use: "kubectl-plugin",
Short: "My root command",
Run: func(cmd *cobra.Command, args []string) {
cmd.SetOutput(streams.ErrOut)
}
}
// You can join all this flags to your root command
flags.AddFlagSet(rootCmd.PersistentFlags())
kubeConfigFlags.AddFlags(flags)
kubeResouceBuilderFlags.AddFlags(flags)
</code></pre>
<p>This is the output:</p>
<pre><code class="language-bash">$ kubectl-plugin --help
My root command
Usage:
kubectl-plugin [flags]
Flags:
--as string Username to impersonate for the operation
--as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
--cache-dir string Default HTTP cache directory (default "/home/gianarb/.kube/http-cache")
--certificate-authority string Path to a cert file for the certificate authority
--client-certificate string Path to a client certificate file for TLS
--client-key string Path to a client key file for TLS
--cluster string The name of the kubeconfig cluster to use
--context string The name of the kubeconfig context to use
-f, --filename strings identifying the resource.
-h, --help help for kubectl-pprof
--insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
--kubeconfig string Path to the kubeconfig file to use for CLI requests.
-n, --namespace string If present, the namespace scope for this CLI request
-R, --recursive Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory. (default true)
--request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
-l, --selector string Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
-s, --server string The address and port of the Kubernetes API server
--token string Bearer token for authentication to the API server
--user string The name of the kubeconfig user to use
</code></pre>
Kubernetes is not for operationsKubernetes it not for operations. It democratize resources and workloads. It can be the solution to bring developers closer to ops. But YAML is not the answer.https://gianarb.it/img/gianarb.png2019-09-18T08:08:27+00:002019-09-18T08:08:27+00:00https://gianarb.it/blog/as-a-developer-i-dont-care-about-kubernetes<p>I work in tech from 8 years. It is not a lot but it is something.
I started as a PHP developer doing CMS with MySQL and things like that.</p>
<p>Where I saw what I was capable to do with a set of API requests to AWS I enjoyed
it and I moved to what people called DevOps probably.</p>
<p>I like communities and people so Docker was everywhere and I became a Docker
Captain for my passion about delivery, and development workflow, containers but
with always developers in mind. That’s what I like do to. Write code.</p>
<blockquote>
<p>The complexity not hidden behind Kubernetes, or not solved by who runs
Kubernetes in your company creates that friction.</p>
</blockquote>
<p>Everyone that was/is in the containers space more or less touched Kubernetes.
I did it, I enjoyed to look at the patterns used by it like control theory,
reconciliation loops and so on.</p>
<p>In the last couple of years I saw a lot of company moving to Kubernetes
and I worked on that path in InfluxData as well. Yes we use Kubernetes obviously!</p>
<p>I have always sawed friction from developers forced to onboard Kubernetes (no
developer will do it otherwise). First because everybody uses YAML and I
think yaml is just the wrong answer for your problem, nothing personal with it.</p>
<p>What developers are happy to do is to <strong>write code</strong> that runs in production and
that gives them good challenge to debug and fix. <strong>write code</strong> is in bold
because that’s what we like most. At least the majority of us.</p>
<p>The complexity not hidden behind Kubernetes, or not solved by who runs
Kubernetes in your company creates that friction.</p>
<p>Running Kubernetes is not hard, we have tutorials, companies, contractors, cloud
providers that can help us out. It is a set of binaries and a database. We run
them since ages! There are a good amounts of them, and they need to be configured,
connected and there are also a lot of different combinations, but that’s fine.
We are used to playing with mobile apps, wordpress plugins and so on.</p>
<p>When I think about myself as a developer I understand why there is this
friction, if I was not passionate about containers at the right time to try out
Kubernetes I probably even had that friction myself.</p>
<p>It does not help me to write better code or to do something different compared
with updating systemd service one by one via <code>ssh</code>. I bet developers working with
Kubernetes in a system under real load will likely get back to <code>ssh</code> to the
servers one by one deploying their new version of the application to have all
the control and visibility they can. That’s what a lot of developers
tries to achieve when I look at them using Kubernetes.</p>
<p>What Kubernetes does very well is democratize ops, it provides a common set of
concepts that we can use to run applications and very good API that abstract the
concrete implementation of containers, VMs, workload, ingress, dns and so on.</p>
<p>We should not west our time trying to run it, we should spend time to make it
usable in our company because that’s we can get from k8s.</p>
<h2 id="my-recipe">my recipe</h2>
<p>I do not have a recipe, a product or something ready to go. But I think there
are two directions I would like to see and to try with a team.</p>
<h3 id="leave-yaml">leave yaml</h3>
<p>YAML is the wrong answer, it is good to make an impact and to write a
document that everyone can read, but your company is not “everybody”, you are
pretty unique. You should use the K8S API. I didn’t have time to make a public
prototype yet but I will do I promise. You should use the language you know
better! I have a lot of experience with go, so my suggestion is to replace yaml
with real code, real function and so on. From Kubernetes 1.16 <code>kubectl diff</code>
runs server side. Sweet!</p>
<h3 id="split-spec-file-by-team">split spec file by team</h3>
<p>It is very easy to end up with a single Kubernetes YAML file that is crazy long.
That file contains everything you run. Across teams, responsabilities and
people. Do not do it. Split it in different files or repositories by team or
application owners.</p>
<p>DevOps, SRE, Sysadmin, reliability, penguins or what ever you call the team that
owns the underline architecture will have the Yaml related to the foundation of
the infrastructure. The content of it is not important for other teams, they
will only write and see what matters to them.</p>
<p>This approaches will decrease complexity for developers making them probably
less worried to screw up part of the infrastructure that is not related to their
work.</p>
<h2 id="conclusion">Conclusion</h2>
<p>If you are a developer please develop good code! If you own Kubernetes in your
company make it to work for your users.</p>
Reactive planning and reconciliation in Gohttps://gianarb.it/img/gianarb.png2019-09-13T08:08:27+00:002019-09-13T08:08:27+00:00https://gianarb.it/blog/reactive-planning-and-reconciliation-in-go<p>Reading my recent posts you can spot my attempt to share what I learned, and
I am still studying around distributed system, control theory and provisioning.</p>
<p>I wrote a quick introduction about why I think <a href="/blog/reactive-planning-is-a-cloud-native-pattern">reactive planning is a cloud
native pattern</a> and I
published an article about <a href="/blog/control-theory-is-dope">control theory</a>, but I
have just scratched the surface of this topic obviously. I have a 470 pages to
read from the book <a href="https://www.amazon.it/Designing-Distributed-Control-Systems-Veli-Pekka/dp/B01FIX9LMG">Designing Distributed Control Systems: A Pattern Language
Approach</a>.
It will take me forever.</p>
<h2 id="introduction">Introduction</h2>
<p>It is easier to explain how much powerful reactive planning is looking at one
example, I wrote it in go, and in this article I am explaining the most
important parts.</p>
<p>Just to summarize, I think resiliency in modern application is crucial and very
hard to achieve in practice, mainly because we need to implement and learn a set
of patterns and rules. When I think about a solid application inside a
microservices environment, or in a high distributed ecosystem my mind drives me
into a different industry. I think about tractors, boilers, and what ever
does not depend on a static state stored inside a database but on a dynamic
source of truth.</p>
<p>When I think about an orchestrator it is clear to me that there is no way to
trust a cache layer in order to understand how many resources (VMs, containers,
pods) are running. We need to check them live because you never know what is
happening to your workload. Those kinds of applications are sensible to latency,
and they require a fast feedback loop.</p>
<p>That’s one of the reason about why when you read about Kubernetes internals you
read about reconciliation loops and informers.</p>
<h2 id="our-use-case">Our use case</h2>
<p>I wrote a small PoC, it is an application that I called
<a href="https://github.com/gianarb/cucumber">cucumber</a>, it is available on GitHub and
you can run it if you like.</p>
<p>It is a CLI tools that provisions a set of resources on AWS. The provisioned
architecture is very simple. You can define a number of EC2 and, they will be
created and assigned to a Route53 record, when the record does not exist the
application will create it.</p>
<p>I learned about how to think about problems like that. At the beginning of my
career the approach was simple, “I know what to do, I need to write a program
that reads the request and does what need to be done”. So you start configuring
the AWS client, parsing the request and making a few API requests.</p>
<p>Everything runs perfectly and you succeed at creating 100 clusters.
Thing starts to be more complicated, you have more resources to provisioning
like load balancers, subnets, security groups and more business logic related to
who can do what. Requests start to be more than 5 at execution and the logic
somethings does not work as linear as it was doing before. At this point you
have a lot of conditions and figuring out where the procedure failed and how to
fix the issue becomes very hard.</p>
<p>This is why my current approach is different when I recognize this kind of
pattern I always start from the current state of the system.</p>
<p>You can question the fact that at the first execution it is obvious that nothing
is there, you can just create what ever needs to be created. And I agree on
that, but assuming that you do not know your starting points drives you to implement
the workflow in a way that is idempotent. When you achieve this goal you can
re-run the same workflow over and over again, if there is nothing to do the
program won’t do anything otherwise it is smart enough to realize what needs to
be done. In this way you can create something called <strong>reconciliation loop</strong>.</p>
<h2 id="reconciliation-loop">Reconciliation loop</h2>
<p>The idea to re-run the procedures over and over assuming you do not know where
you left it is very powerful. Following our example, if the creation flow does
not end because AWS returned a 500 you won’t be stuck in a situation where you
do not know how to end the procedure, you will just wait for the next
re-execution of the flow and it will be able to figure what is already created.
In my example this patterns works great when provisioning the route53 DNS record
because the DNS propagation can take a lot of time and in order to realize if
the DNS record already exists or if there are the right amounts of IPs attached
to it I use the
<a href="https://jameshfisher.com/2017/08/03/golang-dns-lookup/"><code>net.LookupIP</code></a>, it
is the perfect procedure that can take an unknown amount of time to be
addressed.</p>
<h2 id="reactive-planning">Reactive planning</h2>
<p>At the very least reconciliation loop can be explained as a simple <code>loop</code> that
will execute a procedure forever but how do you write a workflow that is able to
understand the state of the system and autonomously make a plan to fix the gap
between current and desired state? This is what reactive planning does and
that’s why control theory is done!</p>
<pre><code class="language-go">// Procedure describe every single step to be executed. It is the smallest unit
// of work in a plan.
type Procedure interface {
// Name identifies a specific procedure.
Name() string
// Do execute the business logic for a specific procedure.
Do(ctx context.Context) ([]Procedure, error)
}
// Plan describe a set of procedures and the way to calculate them
type Plan interface {
// Create returns the list of procedures that needs to be executed.
Create(ctx context.Context) ([]Procedure, error)
// Name identifies a specific plan
Name() string
}
</code></pre>
<p>Let’s start with a bit of Go. <code>Procedure</code> and <code>Plan</code> are the fundamental
interfaces to get familiar with:</p>
<ul>
<li><code>Plan</code> are a collection of <code>Procedures</code>. The <code>Create</code> function is able to
figure out the state of system adding procedures dynamically</li>
<li><code>Procedure</code> are the unit of work, they need to be as small as possible. The
cool part about them is that they can return other procedures (and they can
return other procedures as well) in this way build resilience. If a procedure
returns an error the <code>Plan</code> is marked as failed.</li>
</ul>
<pre><code class="language-go">// Scheduler takes a plan and it executes it.
type Scheduler struct {
// stepCounter keep track of the number of steps exectued by the scheduler.
// It is used for debug and logged out at the end of every execution.
stepCounter int
// logger is an instance of the zap.Logger
logger *zap.Logger
}
</code></pre>
<p><code>Plan</code> and <code>Procedure</code> are crucial, but we need a way to execute a plan, it is
called scheduler. The <code>Scheduler</code> has an <code>Execture</code> function that accept a
<code>Plan</code> and it executes it <strong>until there is nothing left to do</strong>. Procedures can
returns other procedures it means that the scheduler needs to recursively
execute all the procedures.</p>
<p>The way the scheduler has to understand where the plan is done if via the number
of steps returned by the <code>Plan.Create</code> function. The scheduler executes every
plan at last twice, if the second time there are not steps left it means that
the first execution succeeded.</p>
<pre><code class="language-go">// Execute accept an plan as input and it execute it until there are not anymore
// procedures to do
func (s *Scheduler) Execute(ctx context.Context, p Plan) error {
uuidGenerator := uuid.New()
logger := s.logger.With(zap.String("execution_id", uuidGenerator.String()))
start := time.Now()
if loggableP, ok := p.(Loggable); ok {
loggableP.WithLogger(logger)
}
logger.Info("Started execution plan " + p.Name())
s.stepCounter = 0
for {
steps, err := p.Create(ctx)
if err != nil {
logger.Error(err.Error())
return err
}
if len(steps) == 0 {
break
}
err = s.react(ctx, steps, logger)
if err != nil {
logger.Error(err.Error(), zap.String("execution_time", time.Since(start).String()), zap.Int("step_executed", s.stepCounter))
return err
}
}
logger.Info("Plan executed without errors.", zap.String("execution_time", time.Since(start).String()), zap.Int("step_executed", s.stepCounter))
return nil
}
</code></pre>
<p>The <code>react</code> function implements the recursion and as you can see is the place
where the procedures get executed <code>step.Do</code>.</p>
<pre><code class="language-go">// react is a recursive function that goes over all the steps and the one
// returned by previous steps until the plan does not return anymore steps
func (s *Scheduler) react(ctx context.Context, steps []Procedure, logger *zap.Logger) error {
for _, step := range steps {
s.stepCounter = s.stepCounter + 1
if loggableS, ok := step.(Loggable); ok {
loggableS.WithLogger(logger)
}
innerSteps, err := step.Do(ctx)
if err != nil {
logger.Error("Step failed.", zap.String("step", step.Name()), zap.Error(err))
return err
}
if len(innerSteps) > 0 {
if err := s.react(ctx, innerSteps, logger); err != nil {
return err
}
}
}
return nil
}
</code></pre>
<p>All the primitives described in this section are in their go module called
<a href="https://github.com/gianarb/planner">github.com/gianarb/planner</a> that you can
use. Other than what showed here the scheduler supports context cancellation and
deadline. In this way you can set a timeout for every execution.</p>
<p>One of the next big feature I will develop is a reusable reconciliation loop for
plans. In cucumber, it is very raw. Just a goroutine and a WaitGroup to keep the main
process up:</p>
<pre><code>go func() {
logger := logger.With(zap.String("from", "reconciliation"))
scheduler.WithLogger(logger)
for {
logger.Info("reconciliation loop started")
if err := scheduler.Execute(ctx, &p); err != nil {
logger.With(zap.Error(err)).Warn("cucumber reconciliation failed.")
}
time.Sleep(10 * time.Second)
logger.Info("reconciliation loop ended")
}
}()
</code></pre>
<p>But this is too simple and it does not work in a distributed environment where
only one process should run the reconciliation and not all the replicas.</p>
<p>I wrote this code to help myself to internalize and explain what reactive
plans means. And also because I think the go community has a lot of great tools
that make uses of this concept like Terraform, Kubernetes but there are not low
level or simple to understand pieces of code. The next chapter describes how to
write your own control plan using reactive planning.</p>
<h2 id="theory-applied-to-cucumber">Theory applied to cucumber…</h2>
<p>Let’s start looking at the <code>main</code> function:</p>
<pre><code class="language-go">p := plan.CreatePlan{
ClusterName: req.Name,
NodesNumber: req.NodesNumber,
DNSRecord: req.DNSName,
HostedZoneID: hostedZoneID,
Tags: map[string]string{
"app": "cucumber",
"cluster-name": req.Name,
},
}
scheduler := planner.NewScheduler()
scheduler.WithLogger(logger)
if err := scheduler.Execute(ctx, &p); err != nil {
logger.With(zap.Error(err)).Fatal("cucumber ended with an error")
}
</code></pre>
<p>In cucumber there is only one Plan the <code>CreationPlan</code>. We create it based on the
YAML file that contains the requested cluster. For example:</p>
<pre><code class="language-yaml">name: yuppie
nodes_num: 3
dns_name: yeppie.pluto.net
</code></pre>
<p>And it gets executed by the scheduler. As you can see if the schedule returns an
error we do not exit, we do not kill the process. We do not panic! Because we
know that things can break and our code is designed to break just a little
and in way it can be recovered.</p>
<p>After the first execution the process spins up a goroutine that is the one I
copied above to explain a raw and simple control loop.</p>
<p>The process stays in the loop until we kill the process.</p>
<p>In order to test the reconciliation you can try to remove one or more EC2 or the
DNS record, watching the logs you will see how inside the loop the scheduler
executes the plan and reconcile the state of the system in AWS with the one you
described in the YAML.</p>
<pre><code class="language-bash">CUCUMBER_MODE=reconcile AWS_HOSTED_ZONE=<hosted-zone-id> AWS_PROFILE=credentials CUCUMBER_REQUEST=./test.yaml go run cmd/main.go
</code></pre>
<p>This is the command I uses to start the process.</p>
<p>The steps I wrote in cucumber are not many and you can find them inside
<code>./cucumber/step</code>:</p>
<ol>
<li>create_dns_record</li>
<li>reconcile_nodes</li>
<li>run_instance</li>
<li>update_dns_record</li>
</ol>
<p><code>run_instance</code> for example is very small, it interacts with AWS via the go-sdk
and it creates an EC2:</p>
<pre><code class="language-go">package step
import (
"context"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/gianarb/planner"
"go.uber.org/zap"
)
type RunInstance struct {
EC2svc *ec2.EC2
Tags map[string]string
VpcID *string
SubnetID *string
logger *zap.Logger
}
func (s *RunInstance) Name() string {
return "run-instance"
}
func (s *RunInstance) Do(ctx context.Context) ([]planner.Procedure, error) {
tags := []*ec2.Tag{}
for k, v := range s.Tags {
if k == "cluster-name" {
tags = append(tags, &ec2.Tag{
Key: aws.String("Name"),
Value: aws.String(v),
})
}
tags = append(tags, &ec2.Tag{
Key: aws.String(k),
Value: aws.String(v),
})
}
steps := []planner.Procedure{}
instanceInput := &ec2.RunInstancesInput{
ImageId: aws.String("ami-0378588b4ae11ec24"),
InstanceType: aws.String("t2.micro"),
//UserData: &userData,
MinCount: aws.Int64(1),
MaxCount: aws.Int64(1),
SubnetId: s.SubnetID,
TagSpecifications: []*ec2.TagSpecification{
{
ResourceType: aws.String("instance"),
Tags: tags,
},
},
}
_, err := s.EC2svc.RunInstances(instanceInput)
if err != nil {
return steps, err
}
return steps, nil
}
</code></pre>
<p>As you can see the unique situation where I return an error is if the
<code>ec2.RunInstance</code> fails, but only because this is a simple implementation.
Moving forward you can replace the return of that error with other steps, for
example you can terminate the cluster and cleanup, in this way you won’t left
broken cluster around, or if you try other steps to recover from that error
leaving at the next executions (from the reconciliation loop) to end the
workflow.</p>
<p>From my experience reactive planning makes refactoring or development very
modular, as you can see you do not need to make all the flow rock solid since
day one, because it is very time-consuming, but you always have a clear
entrypoint for future work. Everywhere you return or log an error can be
replaced at some point with steps, making your flow rock solid from the
observation you do from previous run.</p>
<p>The <code>reconcile_nodes</code> is another interesting steps. Because <code>run_insance</code> only
calls AWS and it creates one node, but as you can image we need to create or
terminate a random amount of them depending on the current state of the system.</p>
<ol>
<li>if you required 3 EC2 but there are zero of them you need to run 3 new nodes</li>
<li>if there are 2 nodes but your required 3 we need 1 more</li>
<li>if there are 56 nodes but you required 3 of them we need to terminate 63 EC2s</li>
</ol>
<p>The <code>reconcile_nodes</code> procedures makes that calculation and returns the right
steps:</p>
<pre><code class="language-go">package step
import (
"context"
"github.com/aws/aws-sdk-go/service/ec2"
"go.uber.org/zap"
"github.com/gianarb/planner"
)
type ReconcileNodes struct {
EC2svc *ec2.EC2
Tags map[string]string
VpcID *string
SubnetID *string
CurrentNumber int
DesiredNumber int
logger *zap.Logger
}
func (s *ReconcileNodes) Name() string {
return "reconcile-node"
}
func (s *ReconcileNodes) Do(ctx context.Context) ([]planner.Procedure, error) {
s.logger.Info("need to reconcile number of running nodes", zap.Int("current", s.CurrentNumber), zap.Int("desired", s.DesiredNumber))
steps := []planner.Procedure{}
if s.CurrentNumber > s.DesiredNumber {
for ii := s.DesiredNumber; ii < s.CurrentNumber; ii++ {
// TODO: remove instances if they are too many
}
} else {
ii := s.CurrentNumber
if ii == 0 {
ii = ii + 1
}
for i := ii; i < s.DesiredNumber; i++ {
steps = append(steps, &RunInstance{
EC2svc: s.EC2svc,
Tags: s.Tags,
VpcID: s.VpcID,
SubnetID: s.SubnetID,
})
}
}
return steps, nil
}
</code></pre>
<p>As you can see I have only implemented the <code>RunInstnace</code> step, and there is a
<code>TODO</code> left in the code, it means that scale down does not work for now.
It returns the right steps required to matches the desired state, if there are 2
nodes, but we required 3 of them this steps will return only one <code>RunInstance</code>
that will be executed by the scheduler.</p>
<p>Last interesting part of the code is the <code>CreatePlan.Create</code> function. This is
where the magic happens. As we saw the schedulers calls the <code>Create</code> functions
at least twice for every execution and its responsability is to measure the
current state and according to it calculate the required steps to achieve that
we desire. It is a long function that you have in the repo, but this is an idea:</p>
<pre><code class="language-go">resp, err := ec2Svc.DescribeInstances(&ec2.DescribeInstancesInput{
Filters: []*ec2.Filter{
{
Name: aws.String("instance-state-name"),
Values: []*string{aws.String("pending"), aws.String("running")},
},
{
Name: aws.String("tag:cluster-name"),
Values: []*string{aws.String(p.ClusterName)},
},
{
Name: aws.String("tag:app"),
Values: []*string{aws.String("cucumber")},
},
},
})
if err != nil {
return nil, err
}
currentInstances := countInstancesByResp(resp)
if len(currentInstances) != p.NodesNumber {
steps = append(steps, &step.ReconcileNodes{
EC2svc: ec2Svc,
Tags: p.Tags,
VpcID: vpcID,
SubnetID: subnetID,
CurrentNumber: len(currentInstances),
DesiredNumber: p.NodesNumber,
})
}
</code></pre>
<p>The code checks if the number of running instances are equals to the desired
one, if they are different it calls the <code>ReconcileNodes</code> procedure.</p>
<h2 id="conclusion">Conclusion</h2>
<p>This is it! It is a long article but there is code and a repository you can run!
I am enthusiast about this pattern and the work exposed here because I think it
makes it clear and I tried to keep the context as small as possible to stay
focused on the workflow and the design.</p>
<p>Let me know if you will end up using it! Or if you already do how it is going
<a href="https://twitter.com/gianarb">@gianarb</a>.</p>
Control Theory is dopeThis is an introductive article about control theory applied to microservices and cloud computing. It is a very high level overview about control theory driven by what I loved most about it.https://gianarb.it/img/gianarb.png2019-09-04T08:08:27+00:002019-09-04T08:08:27+00:00https://gianarb.it/blog/control-theory-is-dope<p>For the last two years at InfluxData I worked on our custom orchestrator that
empower InfluxCloud v1 to run. I have some talk about it at InfluxDays, but they
are not recorded so, I can’t really post them here, sadly.</p>
<p>If you are thinking: “Why you should write your own orchestrator?”, I have few
answers for you.</p>
<ol>
<li>Back in the day Kubernetes was not so popular, 4 years ago when InfluxCloud
started it was not at least.</li>
<li>We had since the beginning to manage data and state, people still says that
Kubernetes is not for them today, image how it was 4 years ago.</li>
</ol>
<p>Btw now InfluxCloud v2 leverages Kubernetes.</p>
<p>Writing a good orchestrator is super fun! When I started but still today a big
part of it are frustrating and not so good but the one we wrote following
reactive planning and control theory are lovely! This article is an introduction
about Control Theory. <a href="https://twitter.com/goller">Chris Goller</a> Solution
Architect at InfluxData was the first person that told me about how Control
Theory works in theory, and he pushed me to try reactive planning for our
orchestrator.</p>
<p>As Kubernetes contributor I recognized some of those patterns as looking at
shared informers, controller and so on. So I understood since the beginning that
those patterns was everywhere around me!</p>
<p><a href="https://twitter.com/colmmacc">Colm MacCárthaigh</a> from Amazon Web Service with
his talks (like the one posted here) helped me to find resources to read, more
patterns and use cases for it.</p>
<div class="embed-responsive embed-responsive-16by9 col-xs-12 text-center">
<iframe width="560" height="315" src="https://www.youtube.com/embed/O8xLxNje30M" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope;
picture-in-picture" allowfullscreen=""></iframe>
</div>
<h2 id="why-it-works">Why it works</h2>
<p>When I started to work as a Web Developer, designing APIs or websites I had
different challenges to face. To write a solid CRUD you put all your effort
when a request comes to your API, you validate it, apply transformation to
sanitize the input and if it is valid you save it
in your database. You need to build good UX, complex validations systems and so
on. But what lands in the database is right and rock solid.</p>
<p>There are other systems where you do not have a database that tells you what is
right or not. You need to <strong>measure</strong> the current state, <strong>calculate</strong> what
needs to get back to your desired state and you need to <strong>apply</strong> what you
calculated.</p>
<p>Those systems are everywhere:</p>
<ul>
<li>The boiler you have at home to keep the water warm needs to constantly check if the
desired temperature you set is the current one. What it is stored in its
memory is what you desire, not the truth.</li>
<li>The example Colm MacCárthaigh used is the Autoscaler. It keeps checking the
state of your system based on the scalability rules you set. For example if
CPU is over 70% spin up 3 nodes. The autoscaler measures the current state of
your CPUs and when it is over it calculates what needs to be done and it
executes the scale up or down.</li>
<li>When you read Kubernetes documentation is will see reference to Controller,
reconciliation loop, desired state and so on. All of those concepts come from
Control Theory.</li>
</ul>
<p>Orchestrator but more in general big microservices environment do not have the
concept of data locality as we used to have in the past. The data you need can
change continously, and they need to collected from different sources and
combined in order to calculate what needs to be done.</p>
<p>I think this is the main reason about why patterns coming from Control Theory
works well.</p>
<p>If you need to write a program that provisions 3 virtual machines and attach them
to a random DNS record you can approach this problems in 2 ways. You can write a
procedure that:</p>
<ol>
<li>Creates 3 instances.</li>
<li>Takes the public IPs.</li>
<li>Creates the DNS record with the IPs as A record.</li>
</ol>
<p>Another way you have to fix this issue is to start from checking what you have,
making a plan to matches what it is not as you desire. So it will look like
this:</p>
<ol>
<li>Check how many instances there are and mark what you need to do, if there are
2 of them you need one, if there are 5 you need to delete 2, you there are 0
of them you need to create all of them.</li>
<li>Check if the DNS record is already there and how many IPs are assigned to it.</li>
<li>If it does exist you do not need to create it but you need to check if the
IPs assigned to it are the same of the instances, If they are not you need to
reconcile the DNS record fixing the IPs.</li>
<li>The record does not exist? You can create it.</li>
</ol>
<p>If you are wondering how all those checks makes the system more reliable is because
you never know what you already created or what it is already where. Let’s
assume you are on AWS. API requests can fails at the middle of your process and
you need to know where you are. AWS itself can stop or terminate instances, or
some other procedures can do it or for manual mistake.</p>
<p>Approaching the problem in this way allows you to repeat the flow over and over
because it idempotent and at every retry the process will be able to reconcile
any divergence between what you asked for (3 VMs and one DNS record) and what it
is actually running. This process is called reconciliation loop.</p>
<h2 id="101-architecture">101 architecture</h2>
<p>Colm MacCárthaigh highlights three major areas around how a successful Control
Theory implementation looks like:</p>
<ol>
<li>Measurement process</li>
<li>Controller</li>
<li>Actuator</li>
</ol>
<h2 id="measurement-process">Measurement process</h2>
<p>The way you retrieve the current state of the system is crucial in order to have
a low latency. They are crucial in order to calculate what needs to be done
because from the current state your program get different decisions.</p>
<h2 id="controller">Controller</h2>
<p>This section is where I have more experience with. The desired state is stored
and clear usually. You know here to go. You get the measurements and with this
information you need to write a procedure capable of making a plan stating from
your current state to get to the desired one.</p>
<p>I wrote a few weeks ago an introduction about <a href="https://gianarb.it/blog/reactive-planning-is-a-cloud-native-pattern">reactive
planning</a>
it is the way I used to calculate a plan.</p>
<p>I am also preparing a PoC in Golang with actual code you can run and test to
share in practice what means reactive planning.</p>
<h2 id="actuator">Actuator</h2>
<p>It is the part that take a calculated plan, and it executes it. I worked a lot
with schedulers that are able to take a set of steps and execute them one by one
or in parallel based on needs.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Think about one of them problem you have a try to think in a more reactive way,
starting from checking where you are and not from doing things. Reliability and
stability for your code will improve drastically.</p>
Hack your Google Calendar with gcalcliEverybody uses google calendar in a way or another and if you are a Linux with a light desktop manager such as i3 you lack on some commodities like reminders and notifications for your events. I find gcalcli a very good solution for my pain.https://gianarb.it/img/gianarb.png2019-08-26T08:08:27+00:002019-08-26T08:08:27+00:00https://gianarb.it/blog/hack-your-google-calendar-gcalcli<p>I am pretty bad with meetings. I forget about them for a lot of different
reasons, sometime I do not show up even if few minutes earlier my mind briefly
remembered it.</p>
<p>Meetings are not my daily job, and I do not have them with a lot of different
people: IPM with my team, one-to-one with my manager, various stand up. I can
remember the recurrent one pretty well but it is still an annoying a useless
exercise.</p>
<p>When they are not recurrent they are usually out of my small circle of friends
and it gets even worst because I do not like to be late or to miss it! I swear I
am not like that in real life! I am on time and I prefer to be there earlier.</p>
<p>Anyway! Ryan Betts VP Engineer at InfluxData shared a very nice CLI tools called
<a href="https://github.com/insanum/gcalcli">gcalcli</a>. I love CLI tools as much as I
love API! Probably a bit more because they are the perfect glue between server
side and the best UX ever (also known as <strong>my terminal</strong>).</p>
<p><img src="/img/gintonic.jpg" alt="A good gin tonic is great as close as my terminal" /></p>
<p><strong>gcalcli</strong> is a lovely CLI tool that uses the Google Calendar API to help you
to manage your Google Calendar.</p>
<p>You can do a lot of things: list, search, edit, add events and even more.
The <a href="https://github.com/insanum/gcalcli#login-information">authentication is well
documented</a> you need to
create a project on Google Development Platform with Calendar API access. After
that you get your credential and you follow the link I just posted! Super easy.</p>
<p>When you are logged I wrote this system unit and a timer in order to check every
10 minutes if there are upcoming events:</p>
<pre><code>[Service]
SyslogIdentifier=gcalcli-notification
ExecStart=/usr/bin/gcalcli remind
[Install]
WantedBy=multi-user.target
</code></pre>
<pre><code>[Unit]
Description="Send notification for every meetings set for xxxxx@gmail.com"
[Timer]
OnBootSec=0min
OnCalendar=*:0/10
[Install]
WantedBy=timers.target
</code></pre>
<p>The timer runs every 10 minutes this command <code>/usr/bin/gcalcli remind</code>.
<code>remind</code> uses <code>notify-send</code> to show a lovely notification.</p>
<p>I set it up for my working calendar and let me tell you it works great!
For that reason I was looking for a way to support multiple Google account,
because I would like to use it for my personal Google Calendar as well.</p>
<p>There is a global flag for <code>gcalcli</code> called <code>--config-folder</code>, by default it set
te none it creates a config file with credentials and preferences in your home
directory. If you run <code>gcalcli</code> with that parameter set with a different
location:</p>
<pre><code class="language-bash">$ gcalcli --config-folder ~/.gcalclirc-anotheraccount list
</code></pre>
<p>The CLI won’t find the configuration file and it will proceed with a brand-new
authentication and it will create a new file located where specified. Sweet! I
did that trick in order to have the second Google Account configured and I have
created a new unit and timer with the right flags and now I get notification
from everywhere! So far so good!</p>
<p>Ryan allowed me to share a script he hacked called <code>next</code>, I have it in my
<code>bashrc</code></p>
<pre><code class="language-bash">next() {
datetime=$(date "+%Y-%m-%dT%H:%M")
whatwhere=$(gcalcli --calendar name-your-calendar agenda --tsv --details location $datetime 8pm | head -n 1 | awk 'BEGIN {FS = "\t+"} ; {print $5 " " $6}')
re="([[:digit:]]+)"
if [[ $whatwhere =~ $re ]]; then
room="zoommtg://zoom.us/join?confno=${BASH_REMATCH[1]}"
fi
echo "What: '$whatwhere'"
echo "xdg-open $room"
echo "xdg-open $room" | clipc
}
</code></pre>
<p>I use Linux, he uses MacOS, so I changed the script a bit.</p>
<p><code>xdg-open</code> to make it to work with <code>X</code>, <code>next</code> gets the next closer meeting you have in one
particular calendar (<code>name-your-calendar</code> in my case) and it stores on my
clipboard (via <code>clicp</code>) the command to join a zoom channel. It is super when you
are in a hurry, you will join zoom meetings in a second.</p>
<p>If you use <code>gcalcli</code> and you have other tricks let me know via twitter
<a href="https://twitter.com/gianarb">@gianarb</a> because I would like to try them as well!</p>
I am in love with language serversLanguage Servers are a nice way to reuse common features required by editors such as auto complete, formatting, go to definition. This article is a an open letter to share my love for this project with everybodyhttps://gianarb.it/img/gianarb.png2019-07-30T08:08:27+00:002019-07-30T08:08:27+00:00https://gianarb.it/blog/i-am-in-love-with-language-servers<p>Hello everybody! I am writing this article because I had a chat with a friend of
mine <a href="https://twitter.com/walterdalmut">@wdalmut</a>. He is a busy businessman and
vimmer like me.</p>
<p>This article is a quick and practical way to understand why language servers are
fantastic! Because they are!</p>
<p>When I started to use vim, I was developing almost all the time with PHP. PHP is
a tricky language, and back in the day, YouCompleteMe was the way to go to have
some autocomplete. However, as I said, PHP was not an excellent language for
that because the number of files is enormous, and to load all of them to suggest
functions and methods is tricky. Probably it is still like that.</p>
<p>Compared with a couple of years ago, we have more IDE and editors: Atom, VSCode,
Sublime, and many more. All of them to be successful requires the same features:</p>
<ul>
<li>Syntax highlight</li>
<li>autocomplete</li>
<li>Formatting</li>
</ul>
<p>You can see the language serves as a protocol to abstract and reuse those
features, and many more such as go to definition, find all references, show
documentation. Vim is almost like WordPress; there is a plugin for everything;
for example, there is an excellent vim-go plugin to make vim to work smart with
golang. The problem is works for vim and as I said, almost all the editors
need the set of shared features just listed to be usable on a daily base.</p>
<p>The community that builds a language has a lexer a parser, and it can traverse
the AST for the language that it develops. It has the knowledge and all the
building blocks to provide a tool usable by different clients. The way for them
to build something reusable is a language server. The clients are different
editors.</p>
<p>This story is real, and the Golang community develops
<a href="https://github.com/golang/go/wiki/gopls">gopls</a> (it stays for go please), the
Golang language server. I use it with vim, and as a client, I use
<a href="https://github.com/neoclide/coc.nvim">vim-coc</a></p>
<p>vim-go >1.20 works with gopls as well, you need to set it explicitly:</p>
<pre><code>let g:go_def_mode='gopls'
let g:go_info_mode='gopls'
</code></pre>
<p>This article expresses my love for language server, not for Go or vim-go or vim!
Even if I love all of them!</p>
<p>We spent a good amount of time to achieve developer happiness and to boost our
productivity.</p>
<p>There are more tools and developers around here. The killer feature for LSP is
its ability to create communities and to give us the ability to share reusable
code.</p>
<p>Other than the gopls I also use
<a href="https://github.com/sourcegraph/javascript-typescript-langserver">sourcegraph/javascript-typescript-langserver</a>
for JavaScript and TypeScript and <a href="https://github.com/rust-lang/rls-vscode">rust-lang/rls-vscode</a> for rust.</p>
<p>As you can see, rls-vscode looks from the name a VSCode project, but only
because also VSCode supports the language server protocol!</p>
<p>Thanks sourcegraph, microsoft and everybody behind the LSP effort!</p>
When do you need a Site Reliability Engineer?I read everyday more job description looking for SRE. In the meantime I hear and live the frustration about who does not understand what SRE means and hires somebody that won't fit.https://gianarb.it/img/gianarb.png2019-07-05T08:08:27+00:002019-07-05T08:08:27+00:00https://gianarb.it/blog/when-do-you-need-a-site-reliability-engineer<p>I have started to work as a Site Reliability Engineer more than two years ago as
first hired SRE at InfluxData. I survived and learned to all the eras that
every company that onboard a new position live:</p>
<ol>
<li>Lack of knowledge about what the job role means</li>
<li>Adjustment</li>
<li>Growth</li>
<li>Re-adjustment</li>
<li>Repeat</li>
</ol>
<p>You are an SRE not because you care about reliability, everybody cares about
reliability but because the system is too complex to be driven by a person that
also does other things.</p>
<p>There are no differences with any other “first hired” in a company. Even the
first project manager gets hired when the person who was doing that job can’t
make it anymore because the company needs somebody 100% focused on the product.</p>
<p>The Site Reliability Engineer as a role should improve <strong>service</strong> reliability.
Visibility, observability, logging, scalability, instrumentation are all areas
when it should step to serve better tooling to troubleshoot, identify issues.
Because as we all know, even not that complex distributed system are difficult
to debug, this complexity is caused by what it is called partial failure. The
idea that a distributed system will never fail drastically alltogheter, but it
is continuously in a condition of failure mitigated by re-try policies and or
redundancy.</p>
<p>The ability to acknowledge a problem before it will get reported by a customer
improves reliability.</p>
<p>It is not a responsibility for the Site Reliability Engineer to fix the actual
bug in the service, it can. For all those reasons the SRE knows how to code, and
it should modify the application, and it needs to be close to the team that
builds the service, just as every heterogeneous group has who takes care about
the design, UI, deploy, management.</p>
<h2 id="are-they-the-unique-people-on-call">are they the unique people on-call?</h2>
<p>Obviously no. It’s hard to reach a scale where you can manage a sustainable
rotation only with SREs, and every developer is responsible for the code it
ships. If you managed to have a rotation for every service with different
people, all of the teammates should be on-call.</p>
<p>The SREs other than being part of the rotation is the person responsible for the
MTTR (mean time to repair) and the number of false positive. The Site
Reliability Engineer needs to be able to make the MTTR as short as possible, and
the number of false positive as low as it can. They should improve how the
service is monitored, instrumented, and easy to debug.</p>
<h2 id="do-i-need-an-sre-in-every-service-team">do I need an SRE in every service team?</h2>
<p>It is hard to quantify a number, but the SREs needs to have a structure that
gives them time to hang out together and to see each other as a unique team as
well to share knowledge and to avoid the use of too many technologies across the
company. Even more, if the company is not at a gigantic scale in term of the
number of people. The amount of SREs per team depends on now crucial, and
complex reliability for the service is, how big the service team is. You can
share SREs with organizations and services if they are not too big or too
complicated or if the unit itself has excellent reliability skills embedded in.</p>
<h2 id="what-sre-is-not">What SRE is not</h2>
<p>SRE does not replace your ops team; it is not a person with DevOps skills that
knows containers and Kubernetes. It knows cloud, containers, and kubernetes
because it is a pretty new “unicorn” role.</p>
<p>It is a side effect of being a coder that loves to see its code running smoothly
under real load.</p>
Test in production behind slogansHow fast we are capable of instrumenting an application decrease the out of time requires to understand and fix a bug.https://gianarb.it/img/gianarb.png2019-05-27T08:08:27+00:002019-05-27T08:08:27+00:00https://gianarb.it/blog/test-in-production-behind-slogans<blockquote class="tw-align-center twitter-tweet"><p lang="en" dir="ltr">What do we test before
prod? We do our known unknowns -- does it work? (unit tests). does it fail in
ways I can predict?<br /><br />We need to test our unknown unknowns in production
with ✨observability✨. and experiment upon them with chaos engineering! <a href="https://twitter.com/hashtag/VelocityConf?src=hash&ref_src=twsrc%5Etfw">#VelocityConf</a></p>—
Liz Fong-Jones (方禮真) (@lizthegrey) <a href="https://twitter.com/lizthegrey/status/1139273082412027904?ref_src=twsrc%5Etfw">June
13, 2019</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>I got inspired by Liz’s tweet recently, and I am writing this post as a reminder
for everybody. “Test in prod” is a slogan, a trademark. It doesn’t explain all
the concepts behind a sentence as “things bo getter with Coke” hides why or
how. Slogans are great as a quick reminder for more articulated ideas. They
are useful because in one sentence you can recall to more profound contents
inside your brain.</p>
<p><img class="img-fluid" src="/img/coke-slogan.jpg" /></p>
<p>“You” do not test unknown unknowns in production, mainly because you do not know
your unknowns. In production, you as a developer <strong>validate</strong> three kinds of
things:</p>
<ul>
<li>Complicated parts of your system that are not well covered by tests are
working.</li>
<li>Something you are working on and you would like to be sure that it is working
fine, even if it has a unit test, integration tests and so on.</li>
<li>Crucial part of the system that needs to work or your boss will kick your ass,
and you are afraid that you test them even if you just changed a line of CSS.</li>
</ul>
<p>What “test in prod” means is the fact that somebody, a random customer human or
not randomly will trigger an unknown action that will cause an issue. It doesn’t
even be to be triggered, it can be an environmental issue. For example, what
Twitch call <a href="https://blog.twitch.tv/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap-26c2462549a2">“the refresh
storm”</a>
is an excellent example of an environmental issue. When a broadcaster has a
connectivity issue, all the watcher starts to refresh the page multiple times
thinking to solve the problem. As a side effect, the Twitch infrastructure can
suffer about a high number of requests. This is a no-Twitch problem that becomes
a Twitch problem.</p>
<p>We need to learn and onboard tools and mindset that will help us to improve how
fast we can track, record, fix, and learn from an issue. All the question that
matters happens in production, and by consequence, we need to stay focused on
it. I think a lot of people test in prod in some way.</p>
<p>When your laptop starts, but it restarts by itself after some point you have a
problem. You look around, and you notice that your fan doesn’t run anymore. It
is a pretty simple issue to solve and detect. You hear that the fan doesn’t make
any noise, so you replace it.</p>
<p>I am sorry! Everybody got distracted by distributed system, containers, cloud.
90% of our failures if you know how to design a fault tolerance application are
a partial failure! They are a disaster to figure out, understand, and fix! Only
a subset of our system may break, for a subset of customers, but the same part
works correctly for another subgroup, and you need to figure out why! You should
also be able to message that subgroup of customers to say “I am sorry! Shit
happens, we are working on it”, proactively!</p>
<h2 id="conclusion">Conclusion</h2>
<p>“test in prod” means all the things I wrote and probably way more! It is
reasonable to say that nobody can do anything to avoid “test in prod” to happen,
so have fun!</p>
Instrumentation code is a first citizen in a codebaseHow fast we are capable of instrumenting an application decrease the out of time requires to understand and fix a bug.https://gianarb.it/img/gianarb.png2019-05-27T08:08:27+00:002019-05-27T08:08:27+00:00https://gianarb.it/blog/instrumentation-code-is-a-first-citizen-in-a-codebase<p>A few years ago a log was very similar to a printf statement with a message that
in some way we’re trying to communicate to the outside the current situation of
a specific procedure. The format and composition of the message were not
crucial, the main purpose was to make it easy enough to read. A full-text
search engine is capable of tokenizing and indexing every message for easy
lookup and aggregation and it was enough to fix the gap between a human
understanding log and something that a program can parse and visualize. Cloud
Computing and containers changed the way we architect, visualize and deploy
software:</p>
<ol>
<li><strong>The distribution of our applications</strong>. Compared with a more traditional
approach our application runs in the smallest but much-replicated units
(container, pods, ec2 and so on).</li>
<li><strong>The size</strong> of our application (microservices) and by consequence the
interaction between them, over a not perfect communication layer (the
network).</li>
<li>Applications come and go much more frequently because we have automation that
takes care of the number of replicas running inside a system. They are more
<strong>dynamic</strong> and we do not really have a stable identifier as before:
hostnames, IPs change more often.</li>
</ol>
<p>These points increase the importance for us to get applications metrics out from
our code because that’s the language our application speaks. We rely on them in
order to understand what is going on. We need to realize that logs and metrics
have different purposes:</p>
<ul>
<li>To understand what is going right now</li>
<li>To verify what happened in the past (even from a legal perspective)</li>
<li>To compare</li>
</ul>
<p>They are not random printf. All these purposes require methodologies and tools.
This article will stay focused on the first point: “What is going on?” because
it is a question I ask even to myself when I look at the system I wrote or
manage and the answer is a real pain to retrieve. To troubleshoot a system we
need a very dense amount of information “almost in real time” because that’s
when a system is broken “now” and a picture or a sample of older data in order
to compare the current situation with something that we can define as “working”.
We can not really use old data because our codebase changes frequently (because
somebody told us that we can break and develop fast). So there is not a lot of
value at looking at high-density data coming from two weeks ago where the
codebase was different. That’s why time series databases as InfluxDB have data
retention features built in to keep themselves clean.
<a href="https://github.com/kapacitor/influxdb">InfluxDB</a> removes the data after a
certain amount of time, but with
<a href="https://github.com/influxdata/kapacitor">Kapacitor</a> you can aggregate or sample
the data to an older retention policy in order to keep what you need in the
database. Back in the day, I wrote this article about <a href="https://gianarb.it/blog/what-is-distributed-tracing-opentracing-opencensus">Opentracing and
Opencensus</a>.
This is a follow up after another year of working around code instrumentation,
observability, and monitoring.</p>
<p>First of all both of them are vendor-neutral projects that help you instrument
your applications without lock you with a specific provider. It doesn’t really
need to be a bad, evil vendor. If you use the Prometheus client directly in your code,
everywhere, you will be locked to it forever or until you will find the right
time to move over all your codebase. But it sounds like “change your logger”:
something you would like to do magically, one shot without wasting your time.</p>
<p>OpenTracing is 100% for tracing, the problem it solves is about how to
instrument your application to send traces. OpenCensus does the same, plus it
also takes care of metrics.</p>
<p>These two projects have a major issue, they are TWO different projects. They
were not smart enough to agree on the same format and it split the dev community without
any reason, sham of you!. Good for us they will be
<a href="https://medium.com/opentracing/merging-opentracing-and-opencensus-f0fe9c7ca6f0">merged</a>
together at some point to something called OpenTelemetry. Finally!</p>
<p>Another misunderstanding is around how tracers such as Zipkin, Jager, XRay advertise them self as
“opentracing compatible”. When I think about “compatible” I think like a REST
API that follow some rules, and for that reason, the SystemA is compatible with
SystemB and you can change them transparently.</p>
<p>This is not what happens with tracing infrastructure, because you need to
remember that OpenTracing and OpenCensus play in your codebase size, it is not
REST or nothing like that.</p>
<p>Compatibility, in this case, means that the
tracers (Zipkin, Jaeger, AWS X-Ray, NewRelic) ship an OpenTracing compatible
library across many languages that you can change in your codebase in order to
point your application to a different tracer without changing the
instrumentation code you wrote.</p>
<p>NB: OpenCensus has the same goal for metrics as well</p>
<pre><code class="language-javascript">function initTracer(serviceName) {
var config = {
serviceName: serviceName,
sampler: {
type: "const",
param: 1,
},
reporter: {
agentHost: "jaeger-workshop",
logSpans: true,
},
};
var options = {
logger: {
info: function logInfo(msg) {
logger.info(msg, {
"service": "tracer"
})
},
error: function logError(msg) {
logger.error(msg, {
"service": "tracer"
})
},
},
};
return initJaegerTracer(config, options);
}
const tracer = initTracer("discount");
opentracing.initGlobalTracer(tracer);
</code></pre>
<p>This example comes from
<a href="https://github.com/gianarb/shopmany/blob/end/discount/server.js">shopmany</a> a
test e-commerce I wrote. In this case, the <code>tracer</code> is Jaeger, but if you need
to change to Zipkin you can probably use
<a href="https://github.com/DanielMSchmidt/zipkin-javascript-opentracing">zipkin-javascript-opentracing</a></p>
<p>It is important to evaluate an instrumentation library like OpenCensus,
OpenTracing, OpenTelemetry because there is a community that writes and supports
libraries across many languages and tracers. it means that you do not really
need to write your own library, that sounds a bit like too much! I was very
frustrated about the fact that these two libraries was TWO! I can’t wait to see
how the result will look like. How easy it is to instrument an application is a
key value for a company like Honeycomb.io and this sounds like a good reason for
them to have their own instrumentation library
(<a href="https://github.com/honeycombio/beeline-go">go</a>,
<a href="https://github.com/honeycombio/beeline-nodejs">js</a>,
<a href="https://github.com/honeycombio/beeline-ruby">Ruby</a>), and when they started the
ecosystem was different (it is still a mess today as you read) but I hope that
OpenTelemetry will push everybody to just work together because understanding
what is going in production right now is a hard, messy and amazing challenge.</p>
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">it is so nice to see
how two great open source community such as <a href="https://twitter.com/InfluxDB?ref_src=twsrc%5Etfw">@InfluxDB</a> and <a href="https://twitter.com/ntop_org?ref_src=twsrc%5Etfw">@ntop_org</a> can do
togheter. That's how we can solve observability/monitoring challanges all
togheter <a href="https://twitter.com/Chris_Churilo?ref_src=twsrc%5Etfw">@Chris_Churilo</a></p>—
gianarb (@GianArb) <a href="https://twitter.com/GianArb/status/1126107355895214082?ref_src=twsrc%5Etfw">May
8, 2019</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<h2 id="keep-instrumentation">Keep instrumentation</h2>
<p><img src="/img/infinite-loop.png" alt="The infinite symble" /></p>
<p>The ability to instrument an application fast, precisely increase your
troubleshooting capabilities. Fast your iterate on your instrumentation code
faster your will understand what is going on. It is not a one short exercise but
it is something you improve everyday based on what you will learn. But your
ability to learn depends on how well you can read the language that your
applications exposes (let me tell you a secret, it depends on how well you
instrument your code).</p>
<p>More to read:</p>
<ul>
<li><a href="https://medium.com/jaegertracing/jaeger-and-opentelemetry-1846f701d9f2">Jaeger and
OpenTelemetry</a></li>
<li><a href="https://www.honeycomb.io/blog/how-are-structured-logs-different-from-events/">Structured
logs</a></li>
<li><a href="https://gianarb.it/blog/logs-metrics-traces-aggregation">Logs Metrics Traces are uqually
useless</a></li>
</ul>
After two years at InfluxDataTwo years at InfluxData. Feeling, sensations, pain point, what I have learned.https://gianarb.it/img/gianarb.png2019-05-23T08:08:27+00:002019-05-23T08:08:27+00:00https://gianarb.it/blog/two-years-at-influxdata<p>Hello dudes! I am gonna write down some rambling today.</p>
<p>I am writing this article during my flight back to KubeCon 2019 in Barcelona. I
had a great time at our booth speaking with community enthusiasts, customers,
and developers. I had to leave a day before for unknown circumstances (I am
crazy).</p>
<blockquote class="tw-align-center twitter-tweet"><p lang="en" dir="ltr">I figure out just in
time that I am suppose to take my flight now and not Friday as my brain
memorized. I am running to the airport! Bye <a href="https://twitter.com/hashtag/KubeCon?src=hash&ref_src=twsrc%5Etfw">#KubeCon</a>
and sorry for everybody I won't say 👋 to as planned! 😢 See y
soon!</p>— gianarb (@GianArb) <a href="https://twitter.com/GianArb/status/1131468802707939329?ref_src=twsrc%5Etfw">May
23, 2019</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>A few days ago I just realized I am been working at InfluxData for two years, I
think it is time for me to share something with you about my experience over
there.</p>
<h2 id="community-matters">Community matters</h2>
<p>I moved to Dublin 4 years ago for 1.5 years. I was not able to speak in English
and I knew it was an important skill to learn. So I got a job and I move there.</p>
<p>I knew it was not gonna be forever because I love who I left in my hometown and
I like Italy. Luckily for me, I love open source and Dublin has a huge set of
meetups to attend. The Docker Meetup was my favorite one and also the one where
I met <a href="https://twitter.com/tomwillfixit">@tomwillfixit</a> and
<a href="https://twitter.com/jpetazzo">@jpetazzo</a> he pushed me to join the amazing
Captain program at Docker that was just gonna start at that time.</p>
<p>Anyway, I used and shared my InfluxDB love with the world even before moving to
Dublin when I worked at <a href="https://twitter.com/corleycloud">@corleycloud</a> with
Walter and we wrote the <a href="https://github.com/corley/influxdb-php-sdk">PHP
influxdb-sdk</a>.</p>
<p>At some point, as you can hear from this podcast from <a href="https://www.stitcher.com/podcast/the-new-stack-makers/e/60409328?autoplay=true">The New
Stack</a>
I obsessed <a href="https://twitter.com/Chris_Churilo">@Chris_Churilo</a> from Influx so
much that she referred me there as an SRE. I was so excited, my practical
interview toke three hours of troubleshooting a Go application running in
Docker. I still remember the test, it was fun and friendly.</p>
<p>Anyway, that’s how I got here. Open Source, community, new friends and some luck!</p>
<h2 id="timezone-">Timezone ???</h2>
<p>InfluxData is a remote friendly company and I was ready to get back to Italy.
Everybody warned me about the complexity hidden behind remote working, nobody
really told me anything about the fact that all my colleagues will start to work
almost when I am ready to leave! InfluxData is very respectful and I can say
that in two years I can count with one (maybe two) hands the amount of time I
had to open my laptop at some weird time.</p>
<p><em>I love my laptop, I have it open by myself at a weird time as well!</em></p>
<p>But after two years I need to admit that it requires a great effort from both
sides to work with +9 hours folks for so long. You need to be good at reaching
out to them, they need to remember that it is late in your side if they see me
frustrated or tired. But it is fun and you learn a lot about yourself and from
your teammate all around the globe, I think the effort pays back. Everyone
should have the chance to open their mind with cultures that different.</p>
<p>When I joined the people in Europe was not that many, probably 3-4. Now we are
more, my team is not made by myself anymore but there are other 6 people and
<a href="https://twitter.com/gitirabassi">@gitirabassi</a> is in my timezone. It makes
everything way easier.</p>
<h2 id="unicorn-sf-start-up">Unicorn SF start-up</h2>
<p>This is also my first experience in a “unicorn” startup in US/SF. I do now know
if I can define InfluxData as a unicorn startup but every conference I go, even
in South Africa, there are people that use or known InfluxDB. So I bet we
are <strong>pretty unicorn</strong>. Since I joined I think we are at least x4 more
people and we are still growing <a href="https://grnh.se/97725b851">(We are hiring)</a>. It is excited and stressful.</p>
<p>There are a lot of roles and teams that I never heard before in my career
because I worked a lot with small companies and I am very happy to hang out and
chat with them when we are face to face to understand how salespeople follow
customers or how the outbound sales team can make thousands of calls a day to
figure out the right person that should hear about what we do.</p>
<p>Almost all the time people are the bottleneck because it is hard to collaborate
in a good way when your work environment keeps changing under your feet. But
that’s how this business works and there is a lot to learn about how to survive
and how it works.</p>
<h2 id="i-am-whatever-i-want-and-thats-awesome">I am whatever I want! And that’s awesome</h2>
<p>I started as a web developer 7 years ago, I moved to automation and devops
because I liked to make people comfortable and confident in deploying their code
in production and as a developer, I knew the pain and I am happy to solve them.</p>
<p>As SRE I helped to develop a custom orchestrator for our SaaS with stateful
workloads and databases. I also enjoyed all the tracing and instrumentation
revolution that <strong>observability</strong> pushes.</p>
<p>I love the people and working from home sometimes brings some loneliness on the
table that’s why we have tech communities. That’s why I organize the CNCF Meetup
in Turin, I do open source and I go to conferences. There are millions of ways
to feel less alone online and offline obviously! Meetups, co-working, friends,
beers, and BBQ.</p>
<p>I think I am a bit tired to work so close to infra, ops, and automation. My
“developer side” is pushing me back to where I started. The code (no not at
all PHP I am sorry).</p>
<p>At InfluxData there are a lot of Golang ROCK starts, so I will look around to
understand what I am happy to hack on!</p>
<h2 id="conclusion">Conclusion</h2>
<p>I have no idea where I am trying to go with this rumblings. Re-reading what I
wrote it looks my way to thanks all the people that over these two years helped
me to grow and get better. I like to think that MAYBE somebody in the same
situation having some hard time will stumble to this article and she/he will
realize that <em>everything will be all right!</em>.</p>
<p>Keep rocking!</p>
Workshop DesignI recently developed a workshop about application instrumentation. I ran it at the CloudConf in Turin. I developed it in open source and I thought it was a nice idea to share more about why I did it and how.https://gianarb.it/img/gianarb.png2019-04-19T08:08:27+00:002019-04-19T08:08:27+00:00https://gianarb.it/blog/workshop-design<p>Hello sweet internet! At this point, you should know that I am far to be a
lovely happy code. I like to share what I learn and to have a chat about what
you are doing. That’s as it is! Feel free to follow my wheeze on
<a href="https://twitter.com/gianarb">twitter/gianarb</a>.</p>
<p>If you don’t know what I am gonna speak about, I can tell you this is another
way to enjoy coding!</p>
<p>Recently a friend of mine that organizes the <a href="https://cloudconf.it">CloudConf</a>
in Turin, Italy asked me if I was able to deliver a workshop. Let me say THE
workshop, 8 hours of chatting with exercises and questions.
I did something like that years ago about AngularJS but hey, this sounds like a
challenge, and I love challenges! So I took it.
<img src="/img/got-your-back.jpg" alt="" />
If you read my recent posts you know I have a passion nowadays:</p>
<ul>
<li><a href="/blog/go-observability-is-for-troubleshooting">Observability is for troubleshooting</a></li>
<li><a href="/blog/high-cardinality-database">You need an high cardinality database</a></li>
<li><a href="/blog/logs-metrics-traces-aggregation">Logs, metrics and traces are equally useless</a></li>
</ul>
<p>The topic was clear, I have called it “Application instrumentation”. Lovely!</p>
<p>I am driven by passion and purpose. My passion for troubleshooting and the
purpose of figuring out what the f happen in production. I was ready to work on
it!</p>
<p><img src="/img/passion-fruit.jpg" alt="" /></p>
<h2 id="workshop">Workshop?</h2>
<p>This article is about how I prepared the workshop and I hope it can help
somebody to avoid the same mistake and also to use some of the material I
developed.</p>
<p>I made everything in open source. There are two new repositories on my GitHub one with fake
e-commerce I made using 4 different programming languages:</p>
<ul>
<li>Golang as frontend proxy with a UI in HTTP/JQuery.</li>
<li>Java to do the most secure part of the e-commerce obviously the payment
service.</li>
<li>NodeJS to get discounts for the items.</li>
<li>PHP to get the list of items currently available.</li>
</ul>
<p>You can find the code on
<a href="https://github.com/gianarb/shopmany">github.com/gianarb/shopmany</a>.</p>
<p>I decided to develop a minimum version of the application in order to have it
reusable for another purpose. It can be used to build a use case for Kubernetes
deployment for example or a CI lesson.</p>
<p>The branch <code>master</code> contains the minimum set of features that I need to have an
application that has some sense. But for example, the services are without logs,
metrics and tracing because they will be added as exercises from the attendees.</p>
<p>If you check out the workshop you will be able to see in the history a commit for
every exercise and applications.</p>
<p>The lessons are available on
<a href="https://github.com/gianarb/workshop-observability">github.com/gianarb/workshop-observability</a>,
every directory is a lesson. The readme contains a couple of information about
what where we are, why we should care and one or more exercise to do in practice
in order to familiarize with the concepts.</p>
<p>The lessons I developed for the purpose of the CloudConf workshop are:</p>
<ol>
<li>lesson1 designing a health check endpoint. Adding a single endpoint is a good
way to familiarize with a new application and there is so much to learn about
how to design a good health check endpoint!</li>
<li>lesson2 is about logging and <a href="https://charity.wtf/2019/02/05/logs-vs-structured-events/">structure
logging</a>. I tried
to pick the most popular logging libraries for the languages. Logging using
JSON format to open the door for future serialization as an event.</li>
<li>lesson3 is about InfluxDB v1 and the TICK stack. The goal was to serve a
monitoring stack that can work with a different structure such as events and
traces.</li>
<li>lesson4 is about tracing. Using Jaeger we instrumented and build a trace for
the application.</li>
</ol>
<p>I have also reported an idea of a possible timeline (the one I used at the
CloudConf):</p>
<p>09.00 Registration and presentation
09.30 - 13.00 Theory</p>
<ul>
<li>Observability vs monitoring</li>
<li>Logs, events, and traces</li>
<li>How a monitoring infrastructure looks like: InfluxDB, Prometheus, Jaeger,
Zipkin, Kapacitor, Telegraf…</li>
<li>Deep dive on InfluxDB and the TICK Stack</li>
<li>Deep dive on Distributed Tracing</li>
</ul>
<p>13.00 - 14.00 Launch
14.00 - 17.00 Let’s make our hands dirty
17.30 - 18.00 Recap, questions and so on</p>
<h2 id="learning-during-the-development">Learning during the development</h2>
<p>I like to prepare slides, posts, and workshop because I learn a lot along the
way about concepts that I usually develop during a long set and frustrating
attempts. Or reading a lot of blog posts, books, code. Writing about it helps me
to put together what I learned developing easy to understand materials.</p>
<p>This workshop was not a special case. It is not clear for me that even if there
is a lot going on with OpenCensus, OpenTracing, and other instrumentation
libraries there is still room for improvement.</p>
<p>Instrumenting an application is not anymore just a matter of adding <code>printf</code>
around the execution of the code. But it is the way we have to write an
application capable of behind debugged and that speaks with the outside in an
understandable way.</p>
<p>The course has two different sections: theory and practice.</p>
<p>The theory went well. I do not have a lot to say about it and for me, it is where
I am most comfortable with because it looks like a long talk.</p>
<p>The practical part was for sure a bit too long and I didn’t have time to walk
all the people over it but the fact that there are all the solutions, the
purpose written down helped them to feel less lonely and everyone can follow the
resolution is it can do the exercise in practice.</p>
<p>This usually happens because of different skills set or for trouble configuring
the environment.</p>
<p><code>git</code> helped me a lot, every commit has a diff that I used to explain the
solution of the lessons. People that were not confidently writing the solution in a
particular language had to just <code>cherry-pick</code> the commit in the language they
didn’t know.</p>
<h2 id="collaboration">Collaboration</h2>
<p>The practical part was designed to be a collaboration between people. IMHO it
helps to feel less “at school” but more as a team that it is something we should
feel more comfortable with at work.</p>
<p>I think it worked but not that well. People were supporting and helping each
other. But I probably need to cut the lessons in a different way. I think I will
remove the <code>influxdb</code> lessons injecting the learning process of only what
matters for the course along the other lessons. Next time and I will develop a
new lesson about how to parse the logs and push them to InfluxDB for example.
(let me know if you would like to help me!)</p>
<h2 id="feedback">Feedback</h2>
<p>I asked them to do a survey before the end of the course in order to help me
get their feeling. There is a lot to do and some of their feedback is part
of this article. But in general, I am happy because I have all the material in
order and this for me was just a first iteration. I hope to make it better, to
grant more feedback from the open source and to run it again! So let me know if
you would like to have me on board!</p>
<h2 id="next">Next</h2>
<p>As I said instrumentation is hard and I still hoping to get an easier solution
across languages. I tried OpenCensus but I didn’t manage to have it running at I
was in the rush so I used Jaeger.</p>
<p>I will develop something about structured logging as I said for sure.</p>
<p>I hope to get a lesson from some as a service provider like HoneyComb for
example.</p>
<h2 id="fun-fact">Fun Fact</h2>
<p>The youngest person in the room was a student in high school! Wow!</p>
Observability is for troubleshootingThe difference between monitoring and observability is the fact that observability is for troubleshooting. And you troubleshoot in any environment not only in production. This article contains how I do observability in one of my application in Go.https://gianarb.it/img/gianarb.png2019-02-28T08:08:27+00:002019-02-28T08:08:27+00:00https://gianarb.it/blog/go-observability-is-for-troubleshooting<p>Monitoring notifies you when something does not work. You get an alert, a slap in
the face based on the priority of the issue. Observability is about
troubleshooting, debugging, “looking around.” You don’t use observability
techniques only when something doesn’t work.</p>
<p>Mainly because you don’t know where it happens, it can be anytime.
You observe during development, locally or in production, anytime.</p>
<p>The ability to use the same observability tools and techniques such as tracing,
log analysis and metrics is a tremendous value. You get used to them day by day
and not only under pressure, during an outage.
I practical trip that I can give you when you are instrumenting an application
is about interconnection. You need a way to connect logs, with traces and with
metrics.</p>
<p>There is nothing too complicated to understand. Every HTTP request has its own
generated ID.</p>
<p>This ID will become the trace ID, and it will be attached to all the logs
generated by that request.
One of my application I instrumented uses
<a href="https://github.com/opentracing/opentracing-go">opentracing/opentracing-to</a> and
<a href="https://github.com/uber-go/zap">uber-go/zap</a> as the logger. I use a middleware
similar to the one provided by the
<a href="https://github.com/opentracing-contrib/go-stdlib/blob/master/nethttp/server.go">opentracing-contrib/go-stdlib</a>.</p>
<p>Inside an HTTP handler, I configure the logger to add the <code>trace_id</code> for every
log:</p>
<pre><code class="language-go">logger := GetLogger().With(zap.String("api.handler", "ping"))
if intTraceId := req.Context().Value("internal_trace_id"); intTraceId != nil {
logger = logger.With(zap.String("trace_id", intTraceId.(string)))
}
</code></pre>
<p>In this way from this point in time the <code>logger</code> will add the trace_id to every line of log.</p>
<p>With this code <code>req.Context().Value("internal_trace_id")</code> I am retrieving the
“trace_id” from the context. In Go every HTTP request has a context attached and
this work because inside the middleware I set the trace_id in the context of the
request and also as HTTP header:</p>
<pre><code class="language-go">// This is a temporary fix until this issue will be addressed
// https://github.com/opentracing/opentracing-go/issues/188
// This works only with Zipkin.
zipkinSpan, ok := sp.Context().(zipkin.SpanContext)
if ok == true && zipkinSpan.TraceID.Empty() == false {
w.Header().Add("X-Trace-ID", zipkinSpan.TraceID.ToHex())
r = r.WithContext(context.WithValue(r.Context(), "internal_trace_id", zipkinSpan.TraceID.ToHex()))
}
</code></pre>
<p>Having the <code>trace_id</code> exposed as header is nice because I can ask and traing
everyone to just grab that parameter when they have issues. Or we can code the
API consumer in a way that takes care about this value when something doesn’t go
as expected.</p>
<p>All these connections are useful to build a context from different sources. This
is the secret for happiness and Welcome to my Wonderland!</p>
<p><img src="/img/alice-observability.jpg" alt="" /></p>
From sequential to parallel with GoFrom a sequence of action to parallelization in Go. Using channels and wait groups from the sync package.https://gianarb.it/img/gianarb.png2019-02-21T08:08:27+00:002019-02-21T08:08:27+00:00https://gianarb.it/blog/go-parallelization-trick<p>Everything starts as a sequence of events. You have a bunch of things to do and
you are not sure how long or hard to manage they will be.</p>
<p>As a pragmatic developer, you go over the list of things, and you make them one
by one. The script runs, it works, and everyone is happy.</p>
<pre><code class="language-go">package main
import (
"fmt"
"log"
"time"
)
func main() {
list := []string{"a", "b", "c", "d", "e", "f", "g", "h", "i", "l"}
for _, v := range list {
v, err := do(v)
if err != nil {
log.Printf("nop")
}
fmt.Println(v)
}
}
func do(s string) (string,error) {
time.Sleep(1*time.Second)
return fmt.Sprintf("%s-%d", s, time.Now().UnixNano()),nil
}
</code></pre>
<p>Let’s execute it:</p>
<pre><code>$ time go run c.go
a-1550742371537033061
b-1550742372537419148
c-1550742373537846015
d-1550742374538086031
e-1550742375538488129
f-1550742376538746707
g-1550742377539047837
h-1550742378539540979
i-1550742379539938404
l-1550742380540339887
real 0m10.174s
user 0m0.149s
sys 0m0.074s
</code></pre>
<p>Until something changes from the outside, the outside world is a terrible place.</p>
<p><img src="https://media.giphy.com/media/124pc9nFq7ZScU/giphy.gif" alt="" /></p>
<p>The list of things to do grows too much, and your program runs too slow to be
competitive, so you start to think about parallelization.</p>
<p>Luckily for you, every action doesn’t depend on anything else, so you don’t need
to stop if one of them fails or even worst you don’t need to do nothing weird,
you skip that, and you log the failure.</p>
<p>There is an easy way to migrate the code about with something that safely runs
in parallel just using some built-in functions in Go like channels and
WaitGroups.</p>
<pre><code class="language-go">package main
import (
"fmt"
"log"
"sync"
"time"
)
func main() {
fmt.Println("Start")
parallelization := 2
list := []string{"a", "b", "c", "d", "e", "f", "g", "h", "i", "l"}
c := make(chan string)
var wg sync.WaitGroup
wg.Add(parallelization)
for ii := 0; ii < parallelization; ii++ {
go func(c chan string) {
for {
v, more := <-c
if more == false {
wg.Done()
return
}
v, err := do(v)
if err != nil {
log.Printf("nop")
}
fmt.Println(v)
}
}(c)
}
for _, a := range list {
c <- a
}
close(c)
wg.Wait()
fmt.Println("End")
}
func do(s string) (string, error) {
time.Sleep(1 * time.Second)
return fmt.Sprintf("%s-%d", s, time.Now().UnixNano()), nil
}
</code></pre>
<p><code>parallelization</code> should be an external parameter that you can change to
parallelize more or less. With a parallelization factor of 2 the benchmark looks
like:</p>
<pre><code class="language-bash">$ time go run c.go
Start
a-1550742531701829912
b-1550742531701820924
d-1550742532702088077
c-1550742532702180981
e-1550742533702473002
f-1550742533703389899
g-1550742534702714251
h-1550742534703981070
i-1550742535702992582
l-1550742535704308486
End
real 0m5.269s
user 0m0.249s
sys 0m0.078s
</code></pre>
<p>Almost half of the time. Let’s try with 5.</p>
<pre><code class="language-bash">$ time go run c.go
Start
e-1550742633337320607
b-1550742633337280491
c-1550742633337474112
d-1550742633337280481
a-1550742633337298154
h-1550742634338002235
i-1550742634338073772
f-1550742634338033897
g-1550742634338019639
l-1550742634338231670
End
real 0m2.145s
user 0m0.144s
sys 0m0.058s
</code></pre>
<p>I wrote this article because I like how easy it was for this use case to run in
parallel. Based on how complicated your <code>do</code> function is you need to be more
careful.</p>
<p>If your <code>do</code> function calls an external service it can fail, or it can rate
limit you because you are parallelizing too much. But these are all problem that
you can solve increasing the number of safeguards in your code.</p>
<p>Something I learned using this and calling AWS intensively to take snapshots is
the fact that EC2 snapshots happen in the background on AWS, so if you have
thousands of nodes and you call AWS it will rate limit you or you won’t have a
good experience of what happens on the AWS side in reality.</p>
<p>A basic trick is to place a <code>batch delay</code> parameter that sleeps before every
execution</p>
<pre><code class="language-go">v, more := <-c
if more == false {
wg.Done()
return
}
// Sleep here!
v, err := do(v)
if err != nil {
log.Printf("nop")
}
fmt.Println(v)
</code></pre>
<p>This is a very crafty fix but if you catch this problem like me when everything
is failing this is a safe bullet you should try.</p>
<p>Parallelization is fun, but in reality, it increases complexity. Go servers
primitives that are solid foundations but it is not you to instrument your code
well enough to be confident about how it works.</p>
<p>I will write the next chapter about this where I will use opencensus or
opentracing to trace what is going on here!</p>
Short TTL vs Long TTL infrastructure resourceI called this framework "short vs long ttl". GitOps, Infrastructure as code are an hot topic today where the infrastructure is more dynamic and YAML doesn't look like a great solution anymore. In this article I explain a framework I am trying to use to understand when a resource is good to be managed in the old way or not.https://gianarb.it/img/gianarb.png2019-02-14T08:08:27+00:002019-02-14T08:08:27+00:00https://gianarb.it/blog/infra-as-code-short-long-ttl-resource<blockquote class="tw-align-center twitter-tweet"><p lang="en" dir="ltr">I am excited to listen
to a lot of ideas and pains about infra as code and yaml. Everyone is more or
less walking in the same direction. This is what I have in my mind atm. More
will come. Short TTL vs Long TTL resources <a href="https://t.co/XRCOgbB3Rg">https://t.co/XRCOgbB3Rg</a></p>—
pilesOfAbstractions (@GianArb) <a href="https://twitter.com/GianArb/status/1095960644195680257?ref_src=twsrc%5Etfw">February
14, 2019</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Recently I gave a talk at the ConfigManagementCamp about “Infrastructure as
code”
<a href="https://speakerdeck.com/gianarb/cfgmgmtcamp-infrastructure-as-code-should-contain-code">(slides)</a>
and I wrote an article about <a href="/blog/infrastructure-as-real-code">infrastructure as {real}
code</a>.</p>
<p>This post is a follow-up focused on how I identify YAML-friendly resources vs.
something else.</p>
<p>I don’t hate YAML; I think it is a functional specification language, well
supported by a lot of different languages. It works I use it when I need to
write parsable and human-friendly files.</p>
<p><img src="https://media.giphy.com/media/1Mng0gXC5Tpcs/giphy.gif" alt="" /></p>
<p>In infrastructure as code resources mean, almost everything: a subnet, an ec2, a
virtual machine, a DNS record, or a pod.</p>
<p>I reference a single unit you can describe as a <strong>resource</strong>. The name probably
comes from too much CloudFormation specification that I wrote over these years.</p>
<p><strong>Short TTL vs. Long TTL</strong> are two different categories that I use to identify
them. The resources during the evolution of your infrastructure can move between
groups.</p>
<p><strong>Long TTL</strong> resources are the one that doesn’t change much. For example, an AWS
VPC currently doesn’t change. It gets deleted or replaced, but you can not
change the cidr. A Route53 Hosted Zone doesn’t change that often. I am more
confident about using specification languages and traditional tools like
Terraform, CloudFormation or kubectl and YAML for these resources.</p>
<p><strong>Short TTL</strong> resources changes often. Kubernetes deployment and statefulset.
Route53 DNS record in my case or Autoscaling Groups. Manage the lifecycle of
these kinds of resources via YAML requires a lot of automation and file
manipulation that I don’t think it is safe to do. I like a lot more to interact
with the API of my provider, ex. AWS or Kubernetes for them. To avoid programs
that parse and modify YAML or JSON to deploy a slightly different version of a
template I prefer to manipulate actual code. It is what I do every day. I have
testing frameworks, libraries and a lot more patterns to use</p>
<p><img src="/img/shortlongttl.png" alt="" class="img-fluid" /></p>
<p>The location of a resource is dynamic; it can jump from a category to another
based on architectural decisions. One example I have is with AWS AutoScaling
Groups. I like to use them to manage Kubernetes Nodes (workers). At the
beginning when you need a k8s cluster to play with I usually create one
autoscaling group with n replicas of the node. The node as the last command
joins the cluster via kubeadm. Easy like it sounds. In this case, the
autoscaling group is one. It doesn’t change that often. When your use case
becomes more realistic, you need a more complicated topoligies. You need pods to
go on different nodes with more RAM or more CPU or at least you need to labels
or add taints to your cluster to have pods far or closer to others. This means
that you end up having more AutoScaling Group with different configuration and
usually, they go away and get replaced very often with varying versions of
Kubernetes and so on. This dynamicity brought as side effect the request of a
more friendly UX for ops, in our case integrated with the kubectl for example.
That’s when we promoted AutoScaling Groups from a long TTL to a short TTL
resource. We developed a K8S CRD to create autoscaling groups and so on.</p>
<p>The missing part is the <strong>reconciliation</strong> between long TTL and short TTL. As
you can see you end up having YAML or JSON in a repository for the long TTL one
and API requests for the short TTL. It means that you can not tell what’s the
situation for your short TTL resources looking at your repository. You can see
what you run via the kubernetes API, but that’s not what I am looking for. I
think GitOps can fix the issue, but I will write more after more tests.</p>
<p>I tried to make these concepts as clear as possible but let me know what you
think via twitter <a href="https://twitter.com/gianarb">@gianarb</a></p>
Extend Kubernetes via a Shared InformerKubernetes is designed to be extended. There a lot of way to do it via Custom Resource Definition for example. Kubernetes is an event-based architecture and you can use a primitive called Shared Informer to listen on the events triggered by k8s itself.https://gianarb.it/img/k8s.png2019-02-07T08:08:27+00:002019-02-07T08:08:27+00:00https://gianarb.it/blog/kubernetes-shared-informer<p>Kubernetes runs a set of controllers to keep matching the current state of a
resource with its desired state. It can be a Pod, Service or whatever is
possible to control via Kubernetes.
K8S has as core value <em>extendibility</em> to empower operators and applications to
expand its set of capabilities. An event-based architecture where everything
that matters get converted to an event that can be trigger custom code.</p>
<p>When I think about a problem I have that requires to take action when Kubernetes
does something my first target is one of the events that it triggers, example:</p>
<ul>
<li>New Pod Created</li>
<li>New Node Joined</li>
<li>Service Removed
and many, many more.</li>
</ul>
<p>To stay informed about when these events get triggered you can use a primitive
exposed by Kubernetes and the
<a href="https://github.com/kubernetes/client-go">client-go</a> called SharedInformer,
inside the cache package. Let’s see how it works in practice.</p>
<p>First of all as every application that interacts with Kubernetes you need to
build a client:</p>
<pre><code class="language-go">// import "os"
// import corev1 "k8s.io/api/core/v1"
// import "k8s.io/client-go/kubernetes"
// import "k8s.io/client-go/tools/clientcmd"
// Set the kubernetes config file path as environment variable
kubeconfig := os.Getenv("KUBECONFIG")
// Create the client configuration
config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
if err != nil {
logger.Panic(err.Error())
os.Exit(1)
}
// Create the client
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
logger.Panic(err.Error())
os.Exit(1)
}
</code></pre>
<p>As you can see I am commenting the code almost line by line to give you a good
understanding about what is going. Now that you have the client we can create
the SharedInformerFactory. A shared informer listens to a specific resource; the
factory helps you to create the one you need. For this example it lookup the Pod
SharedInformer:</p>
<pre><code class="language-go"> // import v1 "k8s.io/api/core/v1"
// import "k8s.io/client-go/informers"
// import "k8s.io/client-go/tools/cache"
// import "k8s.io/apimachinery/pkg/util/runtime"
// Create the shared informer factory and use the client to connect to
// Kubernetes
factory := informers.NewSharedInformerFactory(clientset, 0)
// Get the informer for the right resource, in this case a Pod
informer := factory.Core().V1().Pods().Informer()
// Create a channel to stops the shared informer gracefully
stopper := make(chan struct{})
defer close(stopper)
// Kubernetes serves an utility to handle API crashes
defer runtime.HandleCrash()
// This is the part where your custom code gets triggered based on the
// event that the shared informer catches
informer.AddEventHandler(cache.ResourceEventHandlerFuncs{
// When a new pod gets created
AddFunc: func(obj interface{}) { panic("not implemented") },
// When a pod gets updated
UpdateFunc: func(interface{}, interface{}) { panic("not implemented") },
// When a pod gets deleted
DeleteFunc: func(interface{}) { panic("not implemented") },
})
// You need to start the informer, in my case, it runs in the background
go informer.Run(stopper)
</code></pre>
<p>Knowing about Shared Informers gives you the ability to extend Kubernetes
quickly. As you can see it is not a significant amount of code, the interfaces
are pretty clear.</p>
<h2 id="use-cases">Use cases</h2>
<p>I used them a lot to write dirty hack but also to complete automation gab a system for example:</p>
<ol>
<li>We used to have a very annoying error during the creation of a Pod with a
persistent volume. It was not a high rate error a restart makes everything to
work as expected. A dirty hack is pretty clear; I automated the manual
process of restarting the pod with that error using a Shared Informer just
like to one I showed you</li>
<li>I am using AWS, and I would like to push some EC2 tags down as kubelet
labels. I use a shared informer but this time to watch when a new node joins
the cluster. From the new node I can get its AWS instanceID (it is a label
itself), and with the AWS API. I can retrieve its tags to identify how to
edit the node itself via Kubernetes API. Everything is part of the <code>AddFunc</code>
in the shared informer itself.</li>
</ol>
<h2 id="complete-example">Complete Example</h2>
<p>This example is a function go program that logs when a new node that contains a
particular tag joins the cluster:</p>
<pre><code class="language-go">package main
import (
"fmt"
"log"
"os"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/util/runtime"
"k8s.io/client-go/informers"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/tools/clientcmd"
)
const (
// K8S_LABEL_AWS_REGION is the key name to retrieve the region from a
// Node that runs on AWS.
K8S_LABEL_AWS_REGION = "failure-domain.beta.kubernetes.io/region"
)
func main() {
log.Print("Shared Informer app started")
kubeconfig := os.Getenv("KUBECONFIG")
config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
if err != nil {
log.Panic(err.Error())
}
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
log.Panic(err.Error())
}
factory := informers.NewSharedInformerFactory(clientset, 0)
informer := factory.Core().V1().Nodes().Informer()
stopper := make(chan struct{})
defer close(stopper)
defer runtime.HandleCrash()
informer.AddEventHandler(cache.ResourceEventHandlerFuncs{
AddFunc: onAdd,
})
go informer.Run(stopper)
if !cache.WaitForCacheSync(stopper, informer.HasSynced) {
runtime.HandleError(fmt.Errorf("Timed out waiting for caches to sync"))
return
}
<-stopper
}
// onAdd is the function executed when the kubernetes informer notified the
// presence of a new kubernetes node in the cluster
func onAdd(obj interface{}) {
// Cast the obj as node
node := obj.(*corev1.Node)
_, ok := node.GetLabels()[K8S_LABEL_AWS_REGION]
if ok {
fmt.Printf("It has the label!")
}
}
</code></pre>
Serverless means extendibilityLooking at the GitHub Actions design and connecting the docs I think I got why serverless is useful. It is a great mechanism to extend platform and SaaS.https://gianarb.it/img/gianarb.png2019-01-22T08:08:27+00:002019-01-22T08:08:27+00:00https://gianarb.it/blog/serverless-means-extendibility<p>I wrote an article about a <a href="/blog/kubernetes-GitHub-action">GitHub Action</a> I
recently created to deploy my code to kubernetes. Very nice. Writing the action
and the post, I realized what serverless is all about. I wrote it in the
incipit of the article, but I think this topic deserves its dedicated post.
Serverless is not yet for web applications. I know some of you will probably
disagree but this is my blog, and that’s why I have one, to write whatever I
like!</p>
<p><img src="/img/brave_dad.png" alt="" /></p>
<p>I used Lambda and API Gateway to distribute two pdf I wrote about <a href="https://scaledocker.com">“how to scale
Docker”</a>, it looks to me way more complicated than a go
daemon. So I wrote them because I got the free tier and because I like to try
new things. There are excellent applications written in that way for example
<a href="https://acloud.guru/">acloud.guru</a> but I am probably not ready for that! My bad</p>
<p>Anyway, I know what I am ready for: We should use serverless to offer
extendibility for our as a service platform.</p>
<p>Good for us distributed system and hispters applications are all based on
events, Kafka and so on. Plus now we have
<a href="https://github.com/opencontainers/runc">runC</a>,
<a href="https://github.com/moby/buildkit">buildkit</a> and a lot of the building blocks
useful to implement a solid serverless implementation.</p>
<p>It is not easy, at scale this is a complicated problem but we are in a better
situation now, and it is a massive improvement from a product perspective:</p>
<ol>
<li>Using containers, we can offer total isolation, and we can take a very
carefully and self defensive approach.</li>
<li>An API already provides extendibility but, you still need to have your server
and to run your application by yourself to enjoy them. With a serverless
approach, it will be much easy for the customer to implement their workflow.</li>
<li>You can ask your customer to share their implementation creating a vibrant
and virtuous ecosystem.</li>
</ol>
<p>You can use a subset of the event that you write in Kafka as a trigger for the
function, VaultDB to store secrets that will be injected inside the service and
so on.</p>
<p><img src="/img/heart.jpg" alt="" /></p>
<p>There is a lot more, but I am excited! Is somebody doing something like that? If
so, let me know <a href="https://twitter.com/gianarb">@gianarb</a>, I would like to chat!</p>
GitHub actions to deliver on kubernetesGitHub recently released a new feature called GitHub Actions. They are a serverless approach to allow developers to run their own code based on what happens to a particular repository. They are amazing for continuous integration and delivery. I used them to deploy and validate kubernetes code.https://gianarb.it/img/gianarb.png2019-01-22T08:08:27+00:002019-01-22T08:08:27+00:00https://gianarb.it/blog/kubernetes-github-action<p>Recently GitHub released a new feature called Actions. To me, it looks like the
best implementation I can think of for serverless. I used AWS Lambda and API
Gateway for some basic API, and I wrote a prototype of an application capable of
running functions using containers called
<a href="https://github.com/gianarb/gourmet">gourmet</a> I don’t buy the fact that it will
make my code easy to manage. At least not to write API or web applications.</p>
<blockquote class="twitter-tweet tw-align-center"><p lang="en" dir="ltr">I used the <a href="https://twitter.com/hashtag/GitHubActions?src=hash&ref_src=twsrc%5Etfw">#GitHubActions</a>
to verify and deploy code to a <a href="https://twitter.com/hashtag/kubernetes?src=hash&ref_src=twsrc%5Etfw">#kubernetes</a>
cluster <a href="https://t.co/nfkjmYKPKs">https://t.co/nfkjmYKPKs</a> I am
impressed about how wonderful this feature is designed and implemented! <a href="https://twitter.com/github?ref_src=twsrc%5Etfw">@Github</a> you
🤘!</p>— :w !sudo tee % (@GianArb) <a href="https://twitter.com/GianArb/status/1087640589838008321?ref_src=twsrc%5Etfw">January
22, 2019</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>That’s why I like what GitHub did because they used serverless for what I think
it is designed for, extensibility.</p>
<p>GitHub Actions just like Lambda functions on AWS are a powerful and managed way
to extend their product straightforwardly.</p>
<p>With AWS Lambda you can hook your code to almost whatever event happens: EC2
creations, termination, route53 DNS record change and a lot more. You don’t need
to run a server, you load your code, and it just works.</p>
<p>Jess Frazelle wrote a blog post about <a href="https://blog.jessfraz.com/post/the-life-of-a-github-action/">“The Life of a GitHub
Action</a>, and I
decided to try something I had my mind since a couple of weeks but it required a
CI server, and it was already too much for me.</p>
<p>Time to time I like the idea to have a kubernetes cluster that I can use for the
testing purpose, so I created a private repository that it is not ready to be
open source because it is a mess with secrets inside and so on.</p>
<p><img src="/img/sorry.jpg" alt="" /></p>
<p>In any case, to give you an idea, this is the project’s folder:</p>
<pre><code>├── .github
│ ├── actions
│ │ ├── deploy
│ │ │ ├── deploy
│ │ │ └── Dockerfile
│ │ └── dryrun
│ │ ├── Dockerfile
│ │ └── dryrun
│ └── main.workflow
└── kubernetes
├── digitalocean.yaml
├── external-dns.yaml
├── micro.yaml
├── namespaces.yaml
├── nginx.yaml
└── openvpn.yaml
</code></pre>
<p>The <code>kubernetes</code> directory contains all the things I would like to install in my
cluster. For every new push on this repository, I would like to check if it can
be applied to the kubernetes cluster with the command <code>kubectl apply -f
./kubernetes --dryrun</code> and when the PR is merged the changes should get applied.</p>
<p>So I created my workflow in <code>.github/main.workflow</code>: ( I left some comment to
make it understandable)</p>
<pre><code>## Workflow defines what we want to call a set of actions.
## For every new push check if the changes can be applied to kubernetes ## using the action called: kubectl dryrun
workflow "after a push check if they apply to kubernetes" {
on = "push"
resolves = ["kubectl dryrun"]
}
## When a PR is merged trigger the action: kubectl deploy. To apply the new code to master.
workflow "on merge to master deploy on kubernetes" {
on = "pull_request"
resolves = ["kubectl deploy"]
}
## This is the action that checks if the push can be applied to kubernetes
action "kubectl dryrun" {
uses = "./.github/actions/dryrun"
secrets = ["KUBECONFIG"]
}
## This is the action that applies the change to kubernetes
action "kubectl deploy" {
uses = "./.github/actions/deploy"
secrets = ["KUBECONFIG"]
}
</code></pre>
<p>The <code>secrets</code> are an array of environment variables that you can use to set
values from the outside. If your account has GitHub Action enabled there is a
new Tag inside the Settings in every repository called “Secrets.”</p>
<p>You can set key-value pairs usable as you see in my workflow. For this example,
I set the <code>KUBECONFIG</code> as the base64 of a kubeconfig file that allows the GitHub
Action to authorize itself to my Kubernetes cluster.</p>
<p>Both actions are similar the first one is in the directory
<code>.github/actions/dryrun</code></p>
<pre><code>├── .github
├── actions
└── dryrun
├── Dockerfile
└── dryrun
</code></pre>
<p>It contains a Dockerfile</p>
<pre><code>FROM alpine:latest
## The action name displayed by GitHub
LABEL "com.github.actions.name"="kubectl dryrun"
## The description for the action
LABEL "com.github.actions.description"="Check the kubernetes change to apply."
## https://developer.github.com/actions/creating-github-actions/creating-a-docker-container/#supported-feather-icons
LABEL "com.github.actions.icon"="check"
## The color of the action icon
LABEL "com.github.actions.color"="blue"
RUN apk add --no-cache \
bash \
ca-certificates \
curl \
git \
jq
RUN curl -L -o /usr/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/v1.13.0/bin/linux/amd64/kubectl && \
chmod +x /usr/bin/kubectl && \
kubectl version --client
COPY dryrun /usr/bin/dryrun
CMD ["dryrun"]
</code></pre>
<p>As you can see to describe an action, you need just a Dockerfile, and it works
the same as in docker. The CMD <code>dryrun</code> is the bash script I copied here:</p>
<pre><code class="language-bash">#!/bin/bash
main(){
echo ">>>> Action started"
# Decode the secret passed by the action and paste the config in a file.
echo $KUBECONFIG | base64 -d > ./kubeconfig.yaml
echo ">>>> kubeconfig created"
# Check if the kubernetes directory has change
diff=$(git diff --exit-code HEAD~1 HEAD ./kubernetes)
if [ $? -eq 1 ]; then
echo ">>>> Detected a change inside the kubernetes directory"
# Apply the changes with --dryrun just to validate them
kubectl apply --kubeconfig ./kubeconfig.yaml --dry-run -f ./kubernetes
else
echo ">>>> No changed detected inside the ./kubernetes folder. Nothing to do."
fi
}
main "$@"
</code></pre>
<p>The second action is almost the same as this one, the Dockerfile is THE same, so
I am not posting it here, but the CMD looks like this:</p>
<pre><code class="language-bash">#!/bin/bash
main(){
# Decode the secret passed by the action and paste the config in a file.
echo $KUBECONFIG | base64 -d > ./kubeconfig.yaml
# Check if it is an event generated by the PR is a merge
merged=$(jq --raw-output .pull_request.merged "$GITHUB_EVENT_PATH")
# Retrieve the base branch for the PR because I would like to apply only PR merged to master
baseRef=$(jq --raw-output .pull_request.base.ref "$GITHUB_EVENT_PATH")
if [[ "$merged" == "true" ]] && [[ "$baseRef" == "master" ]]; then
echo ">>>> PR merged into master. Shipping to k8s!"
kubectl apply --kubeconfig ./kubeconfig.yaml -f ./kubernetes
else
echo ">>>> Nothing to do here!"
fi
}
main "$@"
</code></pre>
<p>That’s everything, and I am thrilled!</p>
<p><img src="/img/party.jpg" alt="" /></p>
<p>There is nothing more to say other than “GitHub actions are amazing!”. They look
well designed since day! The workflow file has a generator that even if I didn’t
use it because I don’t like colors, it seems amazing. The secrets allow us to do
integration with third-party services out of the box and you can use bash to do
whatever you like! Let me know what you use them for on
<a href="https://twitter.com/gianarb">Twitter</a>.</p>
Why I speak at conferencesI am over 50 talks! To celebrate this small, personal achievement I decided to write a post about why I speak at conferences even if I am not an evangelist or a proper DevRel.https://gianarb.it/img/gianarb.png2019-01-15T08:08:27+00:002019-01-15T08:08:27+00:00https://gianarb.it/blog/why-I-speak-at-conferences<p>I have recently counted the number of conferences listed <a href="/conferences.html">here on my
blog</a>, and I realized that I am over 50 talks! I decided to
write this post about why I have started and why I speak at conferences as
celebration.</p>
<h2 id="community">Community</h2>
<p>Everything started when I learned how to develop. When I left university (it was
not the best accomplishment) I began to work in one of the Lab to build a CMS in
PHP to manage their instruments. It was my first experience ever and the worst
scenario I was a solo-man in that company. Kind of a dangerous first job to me I
that’s why I call my experience a community-driven success. I wrote the
application three times:</p>
<ol>
<li>Spaghetti code until I reached the limit for the codebase.</li>
<li>partially re-wrote it with Classes.</li>
<li>I jumped into IRC, and I discovered the community behind Zend Framework.</li>
</ol>
<p>They helped me to figure out how to write the proper version of it. I am in love
with all this open source people that were there to help a newbie like me, and I
discovered the PHP Meetup in my city, Turin. Thank the people I met during an
event I got my second job in a proper company, with other developers and servers
in the basement!</p>
<p>To summarize, the community gave me a lot since my first day: things to learn,
new friends and mentors, and a job. It is natural to return everything I can.</p>
<p>I gave my first talk at one of the local Meetup about Vagrant in 2013. I heart
about it on some IRC channel; it was not popular in Italy yet. So the perfect
chance to give something back to all the people that helped me.</p>
<p>Today after a couple of years technologies and motivations changed, but this is
how I started. I like to be part of a community, that’s why open source is so
important to me. And I want to share what I do and to learn from other people.</p>
<h2 id="italy-is-too-small">Italy is too small</h2>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">we are
privileged, AND we are hustlers. both true <3</p>— Charity Majors
(@mipsytipsy) <a href="https://twitter.com/mipsytipsy/status/1082010778381635584?ref_src=twsrc%5Etfw">January
6, 2019</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>The side effect about being part of an open source community is <strong>globalization</strong>.
You have teammates from everywhere, and you discover great use cases every day.
The way I look at Computer Science requires new challenges and issues to solve,
and I am not ready to take a nap solving too easy problems. This means that I
need to take risks, I changed a lot of companies, and to do that, in some way
you need to share what you are capable off, you need to put your face out there.</p>
<p>This is a rephase from a recent <a href="https://twitter.com/rakyll/status/1084968619505680387">tweet from
@rakyll</a> or at least how
I interpreted it.</p>
<p>Speaking at conferences is an excellent way to discover what other teams are
doing and to meet smart dudes that can turn out to be great mentors.</p>
<h2 id="remote-work">Remote Work</h2>
<p>Two years ago I went back from Dublin, and I decided to try remote working. I
enjoyed it, and it is hard for me to get back at the moment. Working at home
means that I don’t have a lot of social interaction. I am alone for about 8
hours a day, you can fix it moving to a coworking, but conferences or meetups
are a great way to get out! <strong>You don’t need to go far away</strong>, that’s why I run a
<a href="https://www.meetup.com/CNCF-Italy/">meetup in Turin about cloud computing</a> feel
free to let me know when you jump in if you would like to speak.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I am not a developer advocate, and I don’t work in the marketing or sales team
for this reason you need to have support from your company. This is not easy, a
lot of people think that if you have social skills and you are not similar
to a robot you are not a good coder.</p>
<p>I always had managers that helped me to keep going, and I appreciate it. I work
in a startup and time to time is InfluxData that needs technical people at the
conferences that they sponsor, and luckily I love to speak about topics that are
related to what I do at work or to what my company does like monitoring,
observability, distributed system, and clouds, so I am always happy to go!</p>
<p>That’s it! Let me know why you speak at conferences via
<a href="https://twitter.com/gianarb">@twitter</a>, and I hope my experience will help more
people to share their experiences; you are great! I will probably follow up with
other articles about how I approach a conference or an abstract so let me know
if you would like to read it as well!</p>
<h2 id="not-really">not really!</h2>
<p>During the process or writing and digesting this post I realized how important
conferences are for me as person and how sad is that not everyone can enjoy them
as I do even if they would like to do it. There are plenty of reasons but I am
not speaking about laziness here. I am speaking about under represented people
or who can not afford to pay or it is not supported by its company.</p>
<p>Luckily there are a lot of groups that we can support to mitigate this problem
and to figure out new ways to bring more people in and to build a comfortable
and friendly environment for everyone. This is a win for everyone!
<a href="https://twitter.com/ProjAlloy">ProjAlloy</a>,
<a href="https://www.womenwhocode.com/about">WhomanWhoCode</a> accepts donations, but even
if you can not give money our you can look around when you attend a conference
in order to be nice and how to make everyone around you too feel good!</p>
<p><img src="/img/share.jpg" width="20%" style="display:initial;" /></p>
<p>Best Regards,
Gianluca</p>
<p>[1] if you don’t know where to start you can pick a Meetup close to your place!
They are always looking for a speaker and a smaller community can help you to
give your first talk! I usually try my new talks in a meetup too!</p>
<p>[2] Be open during interviews, if you like to speak at conference you need to
convince the new company that it is a valuable skills for them too!</p>
testcontainer library to programmatically provision integration tests in Go with containersProvisioning the environment for integration tests is not easy. You need a flexible strategy to build isolated environment per test and to inject the data you need to verify your assertions. I have ported a popular library from Java to Golang called testcontainers. It wraps the Docker API in order to provide a simple test friendly library that you can use to run containers in test cases.https://gianarb.it/img/gianarb.png2019-01-08T08:08:27+00:002019-01-08T08:08:27+00:00https://gianarb.it/blog/testcontainers-go<p>There are a lot of information in the title I know, but I am not good enough to
make it simple.</p>
<p>Back in the days, I tried to make some contribution to <a href="https://github.com/openzipkin/zipkin">OpenZipkin</a> an open
source tracing infrastructure in Java. I never really worked in that language, and apparently, I failed, but it wasn’t all a waste of time.</p>
<p>OpenZipkin has an excellent integration test suite, and I liked the approach it took to write
integration tests for all the backends it supports MySql, ElasticSeach,
Cassandra.</p>
<p>Provision the integration test environment is complicated even when you do it
wrong:</p>
<ol>
<li>Without a per test isolation.</li>
<li>Without a cleanup process.</li>
<li>Without putting the right effort to have isolated tests.</li>
</ol>
<p>If you try to make integration tests in the right way, you will have a very hard
time, but Zipkin uses a project called
<a href="https://github.com/testcontainers/testcontainers-java">testcontainers-java</a>. It is a library
that wraps the Docker SDK to offer a friendly API to write integration
tests using containers.</p>
<h2 id="why-containers">Why containers</h2>
<p>In 2019 everyone knows the answer, containers are great for integration testings
because they are a lightweight and flexible technology. Docker provides the
architecture that simplifies how you can turn them on and off.</p>
<p>You can spin up a bunch of containers for every integration tests, they will be
fresh new, and you can terminate them at the end of the tests. This increases
isolation a lot, and it makes your tests more stable and easy to reproduce.</p>
<h2 id="golang">Golang</h2>
<p>I develop in Go every day I loved the approach, so I decided to port that
library to Golang and it eventually get moved to the
<a href="https://github.com/testcontainers">testcontainers</a> GitHub
organization under the repository <a href="https://github.com/testcontainers/testcontainers-go">testcontainers/testcontainers-go</a>.</p>
<p>There is a lot to do but I think at this point the API is stable and we have
everything we need to use it. All the rest will be driven by yourself asking for
new features or from contributors that will port more things from the java
project.</p>
<p>This is our “Hello World.”</p>
<pre><code class="language-golang">package main
import (
"context"
"fmt"
"net/http"
"testing"
testcontainers "github.com/testcontainers/testcontainers-go"
)
// TestNginxLatestReturn verifies that a requesto to root returns 200 as status
// code
func TestNginxLatestReturn(t *testing.T) {
ctx := context.Background()
// Request an nginx container that exposes port 80
req := testcontainers.ContainerRequest{
Image: "nginx",
ExposedPorts: []string{"80/tcp"},
}
nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Error(err)
}
// At the end of the test remove the container
defer nginxC.Terminate(ctx)
// Retrieve the container IP
ip, err := nginxC.Host(ctx)
if err != nil {
t.Error(err)
}
// Retrieve the port mapped to port 80
port, err := nginxC.MappedPort(ctx, "80")
if err != nil {
t.Error(err)
}
resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port()))
if resp.StatusCode != http.StatusOK {
t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode)
}
}
</code></pre>
<p>This is a straightforward test, but you can imagine a lot of other use cases. Let’s say that you need to test how your <code>application A</code> interact with an <code>application B</code> that
depends on Redis. You can programmatically build the environment you need in the tests:</p>
<pre><code>// You spin up the Redis container
req := testcontainers.ContainerRequest{
Image: "redis",
ExposedPorts: []string{"6379/tcp"},
}
redisC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Error(err)
}
defer redisC.Terminate(ctx)
ip, err := redisC.Host(ctx)
if err != nil {
t.Error(err)
}
redisPort, err := redisC.MappedPort(ctx, "6479/tcp")
if err != nil {
t.Error(err)
}
// Spin up Application B
appB, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
Env: map[string]string{
"REDIS_HOST": fmt.Sprintf("http://%s:%s", ip, redisPort.Port()),
},
})
if err != nil {
t.Error(err)
}
ipB, err := redisC.Host(ctx)
if err != nil {
t.Error(err)
}
portB, err := redisC.MappedPort(ctx, "8081/tcp")
if err != nil {
t.Error(err)
}
defer appB.Terminate(ctx)
defer redis.Terminate(ctx)
// Now you can use the go function from your application A that interact with
// application B
bclient := appA.NewServiceBClient(ipB, portB)
content, err := bclient.GetKey("my-key")
// Check what you need to check
</code></pre>
<h2 id="programmable-environment-is-the-key">Programmable environment is the key</h2>
<p>I wrote about my relationship with <a href="/blog/infrastructure-as-real-code">infrastructure as
code</a> in a previous article but once again
the fact that you can programmatically build your infrastructure
using real code is the key for all this flexibility.</p>
<p>As plus for integration tests, you can build the environment you need from inside the test case itself, this ability provides significant control over it.</p>
<p>If you need to worm up etcd with some data, you spin up the etcd container and
you push your data using the traditional Go <a href="https://github.com/etcd-io/etcd/tree/master/client">etcd client</a>:</p>
<pre><code>// Spin up Etcd
req := testcontainers.ContainerRequest{
Image: "quay.io/coreos/etcd:latest",
ExposedPorts: []string{"2379/tcp"},
}
etcdC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Error(err)
}
defer etcdC.Terminate(ctx)
ip, err := etcdC.Host(ctx)
if err != nil {
t.Error(err)
}
etcdPort, err := redisC.MappedPort(ctx, "2379/tcp")
if err != nil {
t.Error(err)
}
// Configure the etcd client
cfg := client.Config{
Endpoints: []string{"http://" + ip + ":" + etcdPort},
Transport: client.DefaultTransport,
// set timeout per request to fail fast when the target endpoint is unavailable
HeaderTimeoutPerRequest: time.Second,
}
c, err := client.New(cfg)
if err != nil {
log.Fatal(err)
}
kapi := client.NewKeysAPI(c)
// Set the key foo
resp, err := kapi.Set(context.Background(), "/foo", "bar", nil)
</code></pre>
<p>I wrote this article because after a few weeks of coding and revisions I have
finally tagged
<a href="https://github.com/testcontainers/testcontainers-go/releases/tag/0.0.1"><code>v0.0.1</code></a>
and the library is ready to be tried we need feedback and feature requests to
Prioritize the work to do. So feel free to try it and to open GitHub
<a href="https://github.com/testcontainers/testcontainer-go/issues">issues</a>.</p>
Infrastructure as (real) codeInfrastructure as code today is wrong. Tools like Chef, Helm, Salt, Ansible uses a template engine to make YAML or JSON way to smarter, but comparing this solution with a proper coding language you always miss something. GitOps forces you to stick your infrastructure code in a git repository this is good. But infrastructure as code is way more.https://gianarb.it/img/gianarb.png2018-12-31T08:08:27+00:002018-12-31T08:08:27+00:00https://gianarb.it/blog/infrastructure-as-real-code<p>I got different signals from the internet around the topic infrastructure as
code. I worked with a lot of configuration management tools: Chef, Ansible,
Salt. All of them are good and bad almost in the same way, for me it is mainly a
boring syntax switch between them. That’s one of the reasons I have a repulsion
for these kind of tools. This year at InfluxData we moved to Kubernetes, and I
had the chance to see how a migration like that works, and the unique
privileges to work with my collagues to design how the end result looks
like, even if it is a never ending work in progress based on the feedback
that we get from outself and other teams. So I think at this point I can
try to explain why I think infrastructure as code today doesn’t work.</p>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">I’m
starting to think the industry didn’t get the point of “infrastructure as code”.
That people believe codified infrastructure is checking YAMLs into a git repo is
troubling.</p>— Dan Woods (@danveloper) <a href="https://twitter.com/danveloper/status/1078870433246662656?ref_src=twsrc%5Etfw">December
29, 2018</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Configuration management are not entirely useless, but it is like <a href="https://sizovs.net/2018/12/17/stop-learning-frameworks/">learning a
new framework</a>, there
is always something good to learn but it is just a framework. If you pick the
cooler Javascript one, you will probably get a well-paid job in a startup with
candies and a flexible workplace but I am always interested in learning the
underline architecture and patterns. The reconciliation loop that ReactJS built
to interact with the DOM is pretty nice, or the one that Kubernetes has to
manage all the resources. Architecture, design patterns are well more useful
that syntactic sure that you can get from the framework itself even more when
the “sugar” looks like this:</p>
<pre><code class="language-yaml">- name: "(Install: All OSs) Install NGINX Open Source Perl Module"
package:
name: nginx-module-perl
state: present
when: nginx_type == "opensource"
- name: "(Install: All OSs) Install NGINX Plus Perl Module"
package:
name: nginx-plus-module-perl
state: present
when: nginx_type == "plus"
- name: "(Setup: All NGINX) Load NGINX Perl Module"
lineinfile:
path: /etc/nginx/nginx.conf
insertbefore: BOF
line: load_module modules/ngx_http_perl.so;
notify: "(Handler: All OSs) Reload NGINX"
</code></pre>
<p>The above code is an Ansible script that I took from a randomly from the <a href="https://github.com/nginxinc/ansible-role-nginx/blob/master/tasks/modules/install-perl.yml">nginx
role</a></p>
<pre><code class="language-yaml">piVersion: extensions/v1beta1
kind: Deployment
metadata:
name: { template "drone.fullname" . }}-agent
labels:
app: { template "drone.name" . }}
chart: "{ .Chart.Name }}-{ .Chart.Version }}"
release: "{ .Release.Name }}"
heritage: "{ .Release.Service }}"
component: agent
spec:
replicas: { .Values.agent.replicas }}
template:
metadata:
annotations:
checksum/secrets: { include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }}
{- if .Values.agent.annotations }
{ toYaml .Values.agent.annotations | indent 8 }
{- end }
labels:
app: { template "drone.name" . }}
release: "{ .Release.Name }}"
component: agent
</code></pre>
<p>This is an help chart I took from the <a href="https://github.com/helm/charts/blob/master/stable/drone/templates/deployment-agent.yaml">official GitHub
repository</a>.</p>
<p>To be clear, when I imagine a sweet dessert full of sugar it is way different
compared with what I have pasted above.</p>
<p>Both of them work with a template engine that is capable of rendering a template
that looks like YAML. I will never buy that infrastructure as code doesn’t use
the real code but a serialization language.</p>
<p>If you don’t know why YAML or JSON or HCL these are a set of reasons that you
will hear:</p>
<ul>
<li>The curve to learn YAML, JSON, HCL is way more friendly than a proper language
like Go, Javascript, PHP or whatever.</li>
<li>You don’t have all the utilities that a language provides but only what the
template engine exposes. This should help you and your team to avoid terrible
mistakes.</li>
</ul>
<p>These concerns was reasonably at the beginning, when the DevOps culture started,
but now everyone has a good sense of how to code. We do code review, and we
have a lot more experience around patterns and API to handle infrastructure
provisioning.</p>
<ol>
<li>If you know Kubernetes, it has powerful API that you can leverage to write
automation code, same for cloud provider like AWS, GCP or OpenStack.</li>
<li>Reconciliation loop, informer, Workqueue, Controller and CRDs are concepts
from Kubernetes that you can reuse.</li>
<li>I wrote about <a href="https://gianarb.it/blog/reactive-planning-is-a-cloud-native-pattern">reactive
planning</a>
and its application in cloud.</li>
</ol>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">if people refuse to learn things, fire them.<br />if your management won't fire people for not pulling their weight, quit.<br /><br />ENGINEERS: we live in a golden age of opportunity. please use it while it lasts. <a href="https://t.co/OdB24UNl9X">https://t.co/OdB24UNl9X</a></p>— Charity Majors (@mipsytipsy) <a href="https://twitter.com/mipsytipsy/status/1078799382009470979?ref_src=twsrc%5Etfw">December 28, 2018</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>All the concerns that I raised in favor of <code>YAML, JSON vs. code</code> drives to the
risk of writing bad code, but I think there is no way to “remove bad code.” Even
code that looks good today will look bad tomorrow. Find a way to mitigate the
risk is admirable but I don’t think YAML is the right solution, a code
architecture, the right patterns, testing, documentation and code review are the
way to go.</p>
<p>Today therea are people with the right skills to write good code even around
infrastructure, and if you use real code you will have:</p>
<ul>
<li>A reacher set of libraries and tools based on the language that you will pick.</li>
<li>Unit, integration test frameworks.</li>
<li>Compiling or interpreting an actual language will highlight more syntax errors
that every template engine.</li>
<li>Code is way more fun!</li>
<li>You can import your code and you don’t need to make trick things to join
Kubernetes template together</li>
<li>You can instantiate new objects, apply transformations of them from the same
structure to reuse the code that describes your resources (aws autoscaling
group, kubernetes ingress or whatever).</li>
</ul>
<p>This discussion applied to a real word situation with Kubernetes used not via
YAML but with the Go struct provided by the
<a href="https://github.com/kubernetes/client-go/tree/master/kubernetes/typed/core/v1">kubernetes/client-go</a></p>
<pre><code class="language-yaml">apiversion: apps/v1
kind: deployment
metadata:
name: micro
namespace: micro
labels:
app: micro
component: micro
spec:
replicas: 12
selector:
matchlabels:
app: micro
template:
metadata:
labels:
app: micro
spec:
containers:
- name: microapp
image: gianarb/micro
ports:
- containerport: 8080
env:
- name: SLACK_TOKEN
valuefrom:
secretkeyref:
name: slack
key: token
- name: SLACK_USERNAME
value: "myuser'
resources:
limits:
memory: 128mi
requests:
memory: 100mi
</code></pre>
<p>This YAML translated to Golang:</p>
<pre><code class="language-golang">func newMicroDeployment() *appsv1.Deployment {
return &appsv1.Deployment{
TypeMeta: metav1.TypeMeta{
Kind: "Deployment",
APIVersion: "apps/v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "micro",
Namespace: twodotoh.Namespace,
Labels: map[string]string{
"app": "micro",
"component": "micro",
},
},
Spec: appsv1.DeploymentSpec{
Replicas: pointer.Int32Ptr(12),
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{
"app": "micro",
},
},
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"app": "micro",
},
},
Spec: corev1.PodSpec{
Containers: []corev1.Container{
{
Name: "microapp",
Image: "gianarb/micro",
Ports: []corev1.ContainerPort{
{
ContainerPort: 8080,
},
},
Env: []corev1.EnvVar{
{
Name: "SLACK_TOKEN",
ValueFrom: &corev1.EnvVarSource{
SecretKeyRef: &corev1.SecretKeySelector{
LocalObjectReference: corev1.LocalObjectReference{
Name: "slack",
},
Key: "token",
},
},
},
{
Name: "SLACK_USERNAME",
Value: "myuser",
},
},
},
},
},
},
},
}
}
</code></pre>
<p>You can make the function more flexible passing variables like the number of
replicas for example, or you can write transformation function that looks like
<code>WithDifferentMemoryLimit</code> to apply transformation to your <code>runtime.Object</code>.</p>
<pre><code class="language-golang">deployment := newMicroDeployment()
// You can transform them with utils like:
WithDifferentMemoryLimit("200mi", deployment)
</code></pre>
<p>If you play well will Go packages, and if you structure your code you can have
something like:</p>
<pre><code class="language-golang">apps := []*runtime.Object{}
service := micro.NewKubernetesService()
deployment := micro.NewDeployment()
apps = append(apps, service)
apps = append(apps, deployment)
// Deploy via kubernetes api
</code></pre>
<p>I mean, you have the code now! So you can make all the mistakes you usually do
during your daily job!</p>
<p class="small">Hero image via <a href="https://pixabay.com/en/fractal-complexity-geometry-1758543/">Pixabay</a></p>
You need an high cardinality databaseMonitoring and observability in a dynamic environment on Cloud or Kubernetes is a new challange we are facing and I think the tool that plays a big role is an high cardinality database.https://gianarb.it/img/gianarb.png2018-11-28T08:08:27+00:002018-11-28T08:08:27+00:00https://gianarb.it/blog/high-cardinality-database<p>In order to understand how an application performs you need data. Logs, events,
metrics, traces.</p>
<p>Observability and monitoring are expensive because you need to retrieve all this
data across your system. An architeture these days is not a static rock where
nothing happens and everything stays the same. You don’t have your 10 VPC, with
always the same hostname that you can filter for.</p>
<p>Today you are on cloud, your instances are going up and down based on your load
and it is easier for you to replace an EC2 that troubleshooting a failure.</p>
<p>Containers wrap your application and they makes it easy to deploy, as side
effect you release more often, it means more data.</p>
<p>But the data are useless if you can not get anything good out from them, so they
can be your silver buffet or a big pain, the difference is all made by your
ability to use them to answer your questions or in the ability for your team
aggregate them together in order to build automation with them.</p>
<p>To do all off this you need to manage high cardinality, this is word that sales
team in tech are scary off because nobody will never sell an infinite high
cardinality database, everything has a limit, and the unique solution is not a
product itself but it is more like a mindset developers should have.</p>
<ul>
<li>You need to store the raw data for just the right time, forever is not an
option.</li>
<li>You need to give access to these data across the company in order to build
better aggregation. Build Engineers will probably need data not just from the
CI pipeline but also from your VCS. SRE to understand how a code change behave
in prod they need metrics from servers but also from the CI. Spread the
knowledge</li>
</ul>
<p>The technologies that gives you the ability to interact with a big set of
unstructured data should support an high wrtie troughpoot and smart indexes that
will allow your query engine to lookup for what you need fast enough!</p>
<p>So that’s what I have in my mind when I think about a database that can support
monitoring data.</p>
<p>I am not selling anything mainly because I think a final solution doesn’t exist
yet, I can not really tell you what to buy but you should look around for other
companies at your same scale because everyone has this problem:</p>
<ul>
<li>Facebook has scuba</li>
<li>A lot of people use Cassandra and they looks happy at least for its writing
capabilities.</li>
<li>There are new time series databases releases in a daily based</li>
<li>At InfluxData we obviously use InfluxDB for this purpose</li>
</ul>
<p>The general idea here is that the goal should be to group
data that now are in different sources: NewRelic, InfluxDB, ElasticSearch,
Papertrail in the same place, because it is rare to get
the answer for your question just looking at logs, or metrics, you need an
aggregation or a sample of different data.</p>
<p>This will bring the debugging and troubleshooting capabilities of your team to
the next level, and listen to me, if you are working with a microservices
architecture or with a highly distributed environment you need help from
everything!</p>
Reactive planning is a cloud native patternI discovered how a reactive plan works recently during a major refactoring for a custom orchestrator that we write at InfluxData to serve our SaaS offer. In this article I will explain why I think reactive planning is perfect to build cloud native application like container orchestrators and provisioning tools.https://gianarb.it/img/gianarb.png2018-11-28T08:08:27+00:002018-11-28T08:08:27+00:00https://gianarb.it/blog/reactive-planning-is-a-cloud-native-pattern<p>Probably this title can sound a bit weird to anyone that already know what
reactive plan is and how far it can look from all the cloud-native and
distributed system hipster movement but recently one of my colleagues <a href="https://twitter.com/goller">Chris
Goller</a> pushed this pattern to one of the projects
that we have at <a href="https://influxdata.com">InfluxData</a> and I find it glorious!</p>
<p>“In artificial intelligence, reactive planning denotes a group of techniques for
action selection by autonomous agents. These techniques differ from classical
planning in two aspects. First, they operate in a timely fashion and hence can
cope with highly dynamic and unpredictable environments. Second, they compute
just one next action in every instant, based on the current context.”
(<a href="https://en.wikipedia.org/wiki/Reactive_planning">Wikipedia</a>)</p>
<p>The Wikipedia definition of reactive planning as you can see is perfect to
handle a system where the current status can change very frequently based on
external and unpredictable events.</p>
<p>This is a perfect approach for provisioning/orchestrator tool like Mesos, Cloud
Formation, Kubernetes, Swarm, Terraform. Some of them are working like this
already.</p>
<p>The general idea is that before any action you need a plan because for these
tools an action means: cloud interaction, spin up of resources that cost money.
You need to be proactive avoiding useless execution.</p>
<p>A plan is made of a serious of steps and every step can return other steps if it
needs. The plan is complete when there are no steps anymore. The plan gets
executed at least twice, the second time it should return zero steps because the
first attempts built everything you need, this is the signal that determines its
conclusion. If it keeps returning steps it means that there is something to do
and it tries again.</p>
<p>Let’s start with an example. Think about what Cloud Formation does. You can
declare a set of resources and before to take action it needs to understand what
to do. It is making a plan checking the current state of the system. This first
part makes the flow idempotent and solid because you always start from the
current state of the system. It doesn’t matter if it changes over time because
of somebody that removed one of the resources. If something doesn’t exist it
creates or modify it. Very solid.</p>
<p>Every single step is very small. Let’s take another example like creating a pod
in Kubernetes. When you create a pod there are a lot of actions to do:</p>
<ul>
<li>Validation</li>
<li>Generate the pod id, the pod name</li>
<li>Register the pod to the DNS, you</li>
<li>Store it to etcd
Reach to CNI to configure the network</li>
<li>Reach to docker, container or whatever you use to get the container</li>
<li>Maybe reach to AWS to create a persistent volume</li>
<li>Attach the PV</li>
</ul>
<p>If you try to design all interaction in a single “controller” you will end up
with a lot of * if/else, error handling and so on. Mainly because as you can see
almost every step interact over the network with something: database, DNS, CNI,
docker and so on. So it can fail, it needs circuit breaking, retry policy and
much more complexity.</p>
<p>It is a lot better to design the code where every point is a small step if the
step that reaches docker fails it can return itself as “retry” or it can return
other steps to abort everything and clean up. You will end up with small
reusable (or not that much reusable) steps.</p>
<p>All the steps are combined within a plan, the “PodCreation” plan. There is a
scheduler that takes and execute every step in the plan recursively.</p>
<blockquote>
<p>This freedom allows you to use an incremental approach</p>
</blockquote>
<p>The scheduler as first call a create method for the plan, the create method
checks what to do based on the current state of the system, it is the
responsibility of this function to return no steps when there is nothing to do.</p>
<p>I think this Reactive Planning is one of the best ways to organize the code in a
cloud-native ecosystem for its reactive nature as I said and for the fact that
it forces you to check the state of the system if you don’t do that the plan
will keep executing forever. Obviously, you can use a high-level check to skip
a lot of steps, this requires a balance if the plan you are executing is
critical and frequently used you should check for every step if it requires an
effort that won’t pay back you can implement deepest and preciser checks. You
can check for the PodStatus. If it is running we are good nothing to do. Or you
can check if Docker has a container running and if it has the right network
configuration. If it is running but with no network, you can return the step
that interacts with CNI to set the right interface. This freedom allows you to
use an incremental approach, you start with an easy creation method with checks
for only critical and high-level signal demanding a more solid and sophisticated
set of checks for later, when you will have the best knowledge about where
the system fails.</p>
<p class="small">Hero image via
<a href="https://pixabay.com/en/time-time-management-stopwatch-3222267/">Pixabay</a></p>
You will pay the price of a poor designI read the book Philosophy Software Design from John Ousterhout. It opens my eyes giving me more confidence about how to explain and apply solid design concept in software.https://gianarb.it/img/gianarb.png2018-11-10T08:08:27+00:002018-11-10T08:08:27+00:00https://gianarb.it/blog/price-of-poor-software-design<p>The unique way to avoid the price of a poor design project to replace them in
time. But we all know that replacing a software is not a great idea.</p>
<p>Software should be improved via multiple iterations and that’s why you or your
teammates will read a lot more code compared with the amount written and the of
the story.</p>
<p>In a rush of writing a new project, this sentence will sound wrong, but this is
just one of the phase of it. After that, it will be all about reading and
replacing lines of code one by one and if you poorly design the software
somebody will pay the price of it. You or somebody else, probably somebody else
looking at how quick a developer changes job.</p>
<blockquote>
<p>it is in your hands as a developer to think about design, at
least to save yourself from the darkness</p>
</blockquote>
<p>In a fancy unicorn startup, you will hear that there is no time to think about
design, if you are in tech probably that’s not true, but there is always
a project that nobody cares but it needs to be fixed.</p>
<p>Some other companies don’t have time to think about design because they are
always running against something.</p>
<p><img src="/img/hero/poverty.jpg" alt="" class="img-fluid" /></p>
<p>So, as you can see, it is in your hands as a developer to think about design, at
least to save yourself from the darkness.</p>
<p>Recently I read <a href="https://www.amazon.com/Philosophy-Software-Design-John-Ousterhout/dp/1732102201/">“Philosophy Software
Design”</a>
by John Ousterhout. This book turned out to be <strong>not</strong> a breath of fresh air <strong>for
me</strong> was more like a “breath of consolidation”. Professor John Ousterhout fixed on
paper, in an excellent way what I try to do every day but I was not always able to express.</p>
<p>Why comments are essential, deep API vs shallow classes, information hiding. You
should read it!</p>
<p><strong>Design it twice</strong>. It looks expensive but it is a take away from the book that
I think is the practical key to unlock the door to make our work fun and
healthy.</p>
<p>The first solution can’t be the best one. Even if you are smart enough to design
something that it won’t crash we should make an effort to think about another
solutions, to ask for a review, just as we do for when writing code.</p>
<p>In theory, we will have a great design looking in between of all the other
attempts.</p>
Chaos EngineeringI took part at a panel at the Jazoon conference in Switzerland called Chaos Engineering. Where I had a change to learn about techniques and practices around this topic that even if I new about it I never had the chance to put my head on it. In this article I am summarizing my ideas and what I get mainly around the definition of Chaos Enginnering.https://gianarb.it/img/gianarb.png2018-08-23T08:08:27+00:002018-08-23T08:08:27+00:00https://gianarb.it/blog/chaos-engineer<p>At the <a href="https://jazoon.com/">Jazoon</a> conference in Switzerland, I had the chance
to speak at the Chaos Engineering panel with <a href="https://twitter.com/russmiles">Russ
Miles</a> from <a href="https://chaosiq.io">ChaosIQ</a> and
<a href="https://twitter.com/aaronblohowiak">Aaron P Blohowiak</a> from Netflix.</p>
<p>The organizers put me in the panel probably because “chaos” was part of the title
for the talk I just gave in the morning. I was too curious to mention that I
never did it before, at least on purpose!</p>
<p>So I was really out of my comfort zone dealing with these two folks that know
their shit so well!</p>
<p>I am sure that as Engineers we are part of the Chaos, we create entropy inside the
system during every deploy and even if we have all the tests in the world the
first time it is tough to make it work. But I indeed never associated the
word engineering to chaos. And that’s the real challenge.</p>
<p>So, let’s define Chaos and Engineering altogether.</p>
<p>Let’s start with <code>Chaos</code> because it is the easy one, as I said we as developers create
chaos, distribution creates chaos, and customers create chaos. If somebody tells you
that his production environment is excellent, you should not listen to him,
Production is a nightmare, complicated and painful place. At least if somebody uses it.</p>
<p>And if it is just a bit more complicated than a static site it never works 100%,
the chaos governs it, and that’s where the sentence Engineering becomes
essential.</p>
<p><code>Engineering</code> at least for what I can understand means to be driven by data and
not feeling. So associating these two concepts together you have a powerful way
to measure the chaos.</p>
<p>I think you can’t avoid chaos, so the best way to handle it is to learn from
what it generates in your system to anticipate unpredictable situations.</p>
<p>As developers, ops or devops we are pessimistic about our system, and we know
that it will fail: servers crashes, CoreOS auto updates itself, third party
services stop to work. Usual the answer is to wait for it to happen usually
Friday night.</p>
<p>Chaos Engineering is an exercise, a practice to leverage “unusual but possible”
situations as teaching vector to our system.</p>
<p>It is another tool to achieve resiliency and to test scalability.</p>
<p>Chaos Engineering doesn’t bring down all your production system in an
unrecoverable way. It designs exercises that you and your team will use to
increase your operational experience and confidence.</p>
<p>Observability is a sort of requirement to understand how a chaotic event changes
the “normal” state of your system. But from another point of view a chaotic even
shed some light for a particular part of your system showing up lack of
monitoring and instrumentation.</p>
<p>There open source framework like <a href="https://github.com/chaostoolkit">Chaos-tookit</a>
or famous tools like <a href="https://github.com/Netflix/chaosmonkey">chaos-money</a>.</p>
<p>I will try to start with some very simple example without writing too much
code. I will get out from my system these metrics:</p>
<ol>
<li>Number of requests (probably from ingress/nginx)</li>
<li>The number of requests with status code > 499</li>
<li>Http request latency</li>
</ol>
<p>After that, I will try to simulate an outage removing or scaling down particular
pods (the one that gets all the traffic) and I will look at how the metrics will
change and how long it takes to recover.</p>
OpenMetrics and the future of the prometheus exposition formatThis post explain my point of view around prometheus exposition format and it summarise the next step with OpenMetrics behing supported by CNCF and other big companies.https://gianarb.it/img/gianarb.png2018-08-23T08:08:27+00:002018-08-23T08:08:27+00:00https://gianarb.it/blog/prometheus-openmetrics-expositon-format<p>Who am I to tell you the future about the prometheus exposition format? Nobody!</p>
<p>I was at the PromCon in Munich in August 2018 and I found the conference great!
A lot of use cases about metrics, monitoring and prometheus itself. I work
at InfluxData and we was there as sponsor but I followed a lot of talks and I
had the chance to attend the developer summit the next day with a lot of
promehteus maintainers. Really good conversarsations!</p>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">I just
realized how lucky I was these days having chance to be so welcomed by the <a href="https://twitter.com/hashtag/prometheus?src=hash&ref_src=twsrc%5Etfw">#prometheus</a>
community. I love my work. Thanks <a href="https://twitter.com/juliusvolz?ref_src=twsrc%5Etfw">@juliusvolz</a> <a href="https://twitter.com/TwitchiH?ref_src=twsrc%5Etfw">@TwitchiH</a> <a href="https://twitter.com/tom_wilkie?ref_src=twsrc%5Etfw">@tom_wilkie</a> and
everyone.. I feel regenerated</p>— :w !sudo tee % (@GianArb) <a href="https://twitter.com/GianArb/status/1028414240535793664?ref_src=twsrc%5Etfw">August
11, 2018</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>To be honest my scope few years ago was very different, I was working in PHP
writing webapplication that yes I was deploying but I wasn’t digging to much
around them and I was not smart enough to uderstand that all the pull vs push situation was just all garbage.
Smoke in the eyes that luckily I left behind me pretty soon because I had the
chance to meet smart people that drove me out.</p>
<p>Provide a comfortable way for me to expose and store metrics is a vital
request and the library needs to expose the RIGHT data it doesn’t matter if they
are pushing or pulling.</p>
<p>RIGHT means the best I can get to have more observability from an ops point of
view, but also from a business intelligence prospetive probably just
manipulating again the same data.</p>
<p>It is safe to say that a pull based exposition format is easy to pack together
because it works even if the server that should grab the exposed endpoint is
unavailable or even if nothing will grab them. A push based service will always
create some network noice even if nobody has interest on getting the metrics.</p>
<p>Back in the day we had SNMP but other than being an internet standard the
adoption is not comparable with the prometheus one, if we had how old it is and
how fast prometheus growed the situation gets even worst.</p>
<pre><code>.1.0.0.0.1.1.0 octet_str "foo"
.1.0.0.0.1.1.1 octet_str "bar"
.1.0.0.0.1.102 octet_str "bad"
.1.0.0.0.1.2.0 integer 1
.1.0.0.0.1.2.1 integer 2
.1.0.0.0.1.3.0 octet_str "0.123"
.1.0.0.0.1.3.1 octet_str "0.456"
.1.0.0.0.1.3.2 octet_str "9.999"
.1.0.0.1.1 octet_str "baz"
.1.0.0.1.2 uinteger 54321
.1.0.0.1.3 uinteger 234
</code></pre>
<p>It also started as network exposing format, so it doesn’t express really well
other kind of metrics.</p>
<p>The <a href="https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md">prometheus exposition
format</a>
is extremly valuable and I recently instrumented a legacy application using the
prometheus sdk and my code looks a lot more clean and readable.</p>
<p>At the beginning I was using logs as transport layer for my metrics and time
series but I ended up having a lot of spam in log themself because I was also
streaming a lot of “not logs but metrics” garbage.</p>
<p>The link to the prometheus doc above is the best place to start, here I am just
copy pasting something form there:</p>
<pre><code># HELP http_requests_total The total number of HTTP requests.
# TYPE http_requests_total counter
http_requests_total{method="post",code="200"} 1027 1395066363000
http_requests_total{method="post",code="400"} 3 1395066363000
# Escaping in label values:
msdos_file_access_time_seconds{path="C:\\DIR\\FILE.TXT",error="Cannot find file:\n\"FILE.TXT\""} 1.458255915e9
# Minimalistic line:
metric_without_timestamp_and_labels 12.47
# A weird metric from before the epoch:
something_weird{problem="division by zero"} +Inf -3982045
# A histogram, which has a pretty complex representation in the text format:
# HELP http_request_duration_seconds A histogram of the request duration.
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_bucket{le="0.05"} 24054
http_request_duration_seconds_bucket{le="0.1"} 33444
http_request_duration_seconds_bucket{le="0.2"} 100392
http_request_duration_seconds_bucket{le="0.5"} 129389
http_request_duration_seconds_bucket{le="1"} 133988
http_request_duration_seconds_bucket{le="+Inf"} 144320
http_request_duration_seconds_sum 53423
http_request_duration_seconds_count 144320
# Finally a summary, which has a complex representation, too:
# HELP rpc_duration_seconds A summary of the RPC duration in seconds.
# TYPE rpc_duration_seconds summary
rpc_duration_seconds{quantile="0.01"} 3102
rpc_duration_seconds{quantile="0.05"} 3272
rpc_duration_seconds{quantile="0.5"} 4773
rpc_duration_seconds{quantile="0.9"} 9001
rpc_duration_seconds{quantile="0.99"} 76656
rpc_duration_seconds_sum 1.7560473e+07
rpc_duration_seconds_count 2693
</code></pre>
<p>Think about that not as the prometheus way to grab metrics, but as the language
that your application uses to teach the outside world how does it feels.</p>
<p>It is just a plain text entrypoint over HTTP that everyone can parse and re-use.</p>
<p>For example
<a href="https://www.influxdata.com/time-series-platform/kapacitor/">kapacitor</a> or
<a href="https://www.influxdata.com/time-series-platform/telegraf/">telegraf</a> have
specific ways to parse and extract metrics from that URL.</p>
<p>If you don’t have time to write a parser for that you can use
<a href="https://github.com/prometheus/prom2json">prom2json</a> to get a JSON version of
that.</p>
<p>In Go you can dig a bit more inside that code and reuse some of functions for
example:</p>
<pre><code class="language-go">// FetchMetricFamilies retrieves metrics from the provided URL, decodes them
// into MetricFamily proto messages, and sends them to the provided channel. It
// returns after all MetricFamilies have been sent.
func FetchMetricFamilies(
url string, ch chan<- *dto.MetricFamily,
certificate string, key string,
skipServerCertCheck bool,
) error {
defer close(ch)
var transport *http.Transport
if certificate != "" && key != "" {
cert, err := tls.LoadX509KeyPair(certificate, key)
if err != nil {
return err
}
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert},
InsecureSkipVerify: skipServerCertCheck,
}
tlsConfig.BuildNameToCertificate()
transport = &http.Transport{TLSClientConfig: tlsConfig}
} else {
transport = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: skipServerCertCheck},
}
}
client := &http.Client{Transport: transport}
return decodeContent(client, url, ch)
}
</code></pre>
<p><a href="https://github.com/prometheus/prom2json/blob/master/prom2json.go#L123">FetchMetricsFamilies</a> can be used to get a channel with all the fetched
metrics. When you have the channel you can make what you desire:</p>
<pre><code class="language-go">mfChan := make(chan *dto.MetricFamily, 1024)
go func() {
err := prom2json.FetchMetricFamilies(flag.Args()[0], mfChan, *cert, *key, *skipServerCertCheck)
if err != nil {
log.Fatal(err)
}
}()
result := []*prom2json.Family{}
for mf := range mfChan {
result = append(result, prom2json.NewFamily(mf))
}
</code></pre>
<p>As you can see
<a href="https://github.com/prometheus/prom2json/blob/master/cmd/prom2json/main.go#L42"><code>prom2json</code></a>
converts the result to JSON.</p>
<p>It is pretty fleximple! And it is a common API to read applicatin status. A
common API we all know means automation! Dope automation!</p>
<h2 id="future">Future</h2>
<p>The prometheus exposition format growed in adoption across the board and a
couple of people leaded by <a href="https://twitter.com/TwitchiH">Richard</a> are now pushing
to have this format as new Internet Standard!</p>
<p>The project is called <a href="https://openmetrics.io/">OpenMetrics</a> and it is a Sandbox
project under CNCF.</p>
<p>if you are looking to follow the project here the official repository on
<a href="https://github.com/OpenObservability/OpenMetric">GitHub</a>.</p>
<p>Probably it looks just a political step with no value at all from a tech point of
view but I bet when it will be a standard and not just “the prometheus
exposition” we will start to have routers exposing stats over
<code>http://192.168.1.1/metrics</code> and it will be a lot of fun!</p>
<p>It will be obvious that it is not a <code>only-prometheus</code> feature and this new group
has people from difference companies and backgrounds. So the exposition format
will be probably not just for operational metrics but more generic.</p>
Apps I used during my nomad workingWhen I travel for conferences or now that I work in remote and I am a bit more like a nomad I discovered and learned some good apps that helps me to plan and to combine better work and travel. Some of them are WorkFrom, oBike, Yep, Adobe Scan. let me know yours.https://gianarb.it/img/dna.jpg2018-08-11T10:38:27+00:002018-08-11T10:38:27+00:00https://gianarb.it/blog/nomad-working-apps<p>It is a couple of years now since my first conference and now that I am working
in remote, it is even harder to combine traveling and work.</p>
<p>Mainly because now that I don’t need to go to an office I travel more often, and
usually conference are a better more far away too. It means that I spent more
time, not in my usual workplace.</p>
<p>It is a very challenging and exciting opportunity, and I am glad to live it.
This is the first important things I think. If you are not happy about what you
do, even if it is challenging, you are going to give up.</p>
<p>But other things help me a lot. In general, they are come back to <code>planning.</code> I
feel better when I have the time to look around and to be prepared for the city
I am going to visit. There are a couple of apps that help me with that:</p>
<h2 id="workfrom"><a href="https://workfrom.co/">WorkFrom</a></h2>
<p>It is a community of digital nomad and remote worker. It keeps up to date a
database of pubs, libraries, restaurants, bars where you can work from. It is
very nice, and it has some nice feature like:</p>
<ol>
<li>A map, so you can see what is around you</li>
<li>Net speed measurement inside the app. Other than a detailed description of
the place it also shares (if the person who reviewed the area made it) how
the internet connection is fast and sometimes even the WiFi password, in this
way you don’t need to ask.</li>
<li>As I mentioned it is a community, so there is also a Slack Channel that you
can use to speak with another remote worker.</li>
</ol>
<p>I used it in Berlin, Munich, Copenaghen, Amsterdam and it worked pretty well.</p>
<h2 id="bike-and-other-local-transport-applications-i-am-using">bike and other local transport applications I am using</h2>
<p><a href="https://www.o.bike/it/">oBike</a> as an example because just because I used it
recently in Munich but I think you should always have a look to what the city
uses for bike sharing, for instance, because at least for me even if I love to
walk around take a ride time to time is faster and helpful.</p>
<p>Bonus point a lot of these apps have a free tier in means that you can even use
them for free the first time you visit that city!</p>
<h2 id="yelp">Yelp</h2>
<p>In general, I find <a href="https://www.yelp.com/">Yelp</a> better than TripAdvisor for
what concerns restaurant and place to eat. So when I am not able to spot nothing
good by myself during my walks around the city or when I am looking for a
specific kind of good I use Yelp.</p>
<h2 id="adobe-scan">Adobe Scan</h2>
<p>For this <a href="https://acrobat.adobe.com/us/en/mobile/scanner-app.html">app</a> I need to give
credit to <a href="https://twitter.com/fntlnz">Lorenzo</a> because he showed that app to me
the first time. I use it a lot after a trip when I need to submit the expenses.
It is always a very annoying work to do but at least with this app, I can take a
set of pictures, and it will generate a single pdf ready to be submitted!</p>
<h2 id="revolut">Revolut</h2>
<p>A few years ago when I was working at CurrencyFair I started to test
and play with online banks and <a href="https://revolut.com/r/gianlu1b2">Revolut</a> is
very good when you are traveling around, and you need to manage different
currencies. First of all, I like the idea to use a different card when I buy on
Amazon or when I travel because in case of any trouble I will have a limited
amount of money there. For example in Cube I had my card cloned but I only had
10 Euro, so it was not the first count. (the bank give me the money back in any
case btw).</p>
<p>Plus Revolut has some excellent features to track where and what you spent. You
can label your transfers to easy lookup them when you need to expense or
calculate how much you spent. The exchange commission low compared with the more
traditional banks, so this is a free win!</p>
<p>Let me know on <a href="https://twitter.com/gianarb">Twitter</a> if you have any other
applications to suggest I will be happy to try them next time and maybe to add
them here!</p>
<p><small><a href="https://www.newhdwallpapers.in/natural-hd-wallpapers/himalayas-mountain-series-tibet/" target="_blank">hero img credits</a> </small></p>
FAQ: Distributed tracingTracing is a well known concept in programming but distributed tracing is a revisitation to adapt the concept for a distributed system. This article is an FAQ where I answer common questions I received or I saw around the net about monitoring and distributed tracing.https://gianarb.it/img/dna.jpg2018-07-06T10:38:27+00:002018-07-06T10:38:27+00:00https://gianarb.it/blog/faq-distributed-tracing<p>This article is a write up of a talk that I will give at the
<a href="https://osmc.de">osmc</a> in Germany in November about Distributed tracing.
It is a sequence of questions I got about distributed systems and distributed
monitoring.</p>
<h2 id="why-do-i-need-distributed-tracing">Why do I need distributed tracing?</h2>
<p>It always depends, I find distributed tracing useful in a microservices
environment or more in general when there is a request that flies in a system
crossing different applications, queues or processes.
If you have a problem understanding where a request fails, you need to
<em>follow it</em> in some way and tracing does just that.</p>
<h2 id="how-do-you-follow-a-request">how do you follow a request?</h2>
<p>First of all, we should probably change the name <em>request</em>, it looks too HTTP
oriented, and it is not really what we look for now. In modern application, you
are interested in <em>events</em>. You need to monitor an event:</p>
<ul>
<li>user registration</li>
<li>payment</li>
<li>a bank transaction</li>
<li>send an email</li>
<li>generate an invoice</li>
</ul>
<p>These are all events, and probably in your system, they are distributed not via
HTTP but maybe they go in a queue, or they are broadcasted using Kafka or Redis.
Distributed tracing is all about tracking events. The way to go is to create an
id. Usually, it is called <code>request_id</code> or <code>trace_id</code> and you need a way to
propagate it in your system.</p>
<p>For example, in a queue, you can put the <code>trace_id</code> as part of the payload. Via
HTTP or gRPC you can use Headers.</p>
<p>Your application can take that id, and it can create the span to trace a
particular section.</p>
<h2 id="how-a-trace-looks-like">how a trace looks like?</h2>
<p><img src="/img/trace.jpg" alt="How I image a trace for a distributed tracing app" class="img-fluid" /></p>
<p>In my mind, this is a picture for one trace. Every segment is a span.
So, every span has a trace id, and every span has its own <code>span_id</code>.
You can attach information to every span as key value store. Let’s suppose a
span represent a query in mysql you can put the query as metadata in the span
itslef. In this way you will have a bit more context.</p>
<h2 id="do-we-need-a-standard-for-tracing">do we need a standard for tracing?</h2>
<p>I can’t convince you that interoperability is essential if you already analyzed
the problem and you answered “No” to yourself.
To build a trace you need to agree on something over languages and
applications.
That’s why I think a standard is something you can not avoid, at the end you
will end up having one just for your company.</p>
<h2 id="how-a-tracing-infrastructure-looks">how a tracing infrastructure looks?</h2>
<p><img src="/img/tracing_infra.png" style="width:70%" alt="Sketch of tracing infrastructure." class="img-fluid" /></p>
<p>The applications that are writing traces is not important. Traces is cross
platform and languages. Usually, you
point an app to a tracer. It can be Zipkin, Jaeger or others.</p>
<p>The tracer takes all the traces, and it stores them in a storage. The databases are
usually ElasticSearch, Cassandra, InfluxDB. It depends on which tracer you are
using. They support different databases.</p>
<p>In general traces are high cardinality oriented data, and you can write a lot
of them in a short amount of time. So it is a write-intensive
application.</p>
<p>There are a couple of other pieces that you can add in your tracing
infrastructure:</p>
<ul>
<li>You can add a <em>downsampler</em> to select what to store. If an API request generate
too many traces probably you are interested in storing only a % of them to
decrease pressure on your database. So you can use a simple distributed hash
algorithm on the trace_id to declare what to save or not. A <code>mod</code> on the
<code>trace_id</code> is enough for example.</li>
<li>You can add a <em>collector</em> in front of the tracer. Zipkin support Kafka For
example. In InfluxDB we use telegraf. A collector is usually a stateless
application, it gets all the traces from the applications. It bulks them and
sends them to the tracer. A collector decreases the pressure on the tracer
itself because usually, they work better with a bulk of data. In second if a
tracer go down or you need to update it, the collector is a layer that can
keep the traces for a little bit to give you time to restore the tracer.</li>
</ul>
<h2 id="why-did-i-pick-opentracing">why did I pick opentracing?</h2>
<p>I am an interoperability oriented developer; I think it is essential to avoid
vendor lock-in and embracing a big community like the opentracing one you get
a lot of tools and services already instrumented with this protocol. It makes
my life easy.</p>
<h2 id="can-i-have-a-tracing-infrastructure-on-prem">can I have a tracing infrastructure on-prem?</h2>
<p>You can; there are a couple of tracers open source.</p>
<ul>
<li><a href="https://zipkin.io/">Zipkin</a> is an open source
project in Java started by Twitter.</li>
<li><a href="https://github.com/jaegertracing/jaeger">Jaeger</a> looks a lot like a porting of
it in Golang and Uber makes it.</li>
</ul>
<p>Both of them are open source, and they support different backends like
ElasticSaerch, Cassandra and so on.</p>
<h2 id="there-are-as-a-service-tracing-infrastructure">there are as a service tracing infrastructure?</h2>
<p>There are, NewRelic has an opentracing compatible API, or
<a href="https://lightstep.com/">Lightstep</a> for example. A lot of cloud providers offer
a tracing service. AWS X-RAY or Google Stackdriver.</p>
<h2 id="can-i-store-traces-everywhere">can I store traces everywhere?</h2>
<p>You can, but they are a high cardinality data. The <code>trace_id</code> is usually the
lookup parameter for your queries. It means that it should be indexed, but it
changes for every request. The consequence is a big index.
You need to keep it in mind.</p>
<h2 id="once-you-do-the-tracethen-what">Once you do the trace…then what?</h2>
<p>I left this question as the last one because I read it in the opentracing
mailing list and I think it is a hilarious question.</p>
<p>First of all, you don’t buy a pan and after the fact you start asking yourself
why you have it.</p>
<p>Probably you need to write something, and for that reason, you
buy a pen.</p>
<p>Anyway, I trace my applications because it helps me to understand my environment
over the “distribution complexity.” I can detect what is taking too long and a
trace helps me to understand what to optimize.</p>
<p>Opentracing has a set of standard annotation very useful to detect network
latency between services. You can mark a span as “client send” request for
example. And when the server gets the request, you can mark another span as
“server received.” This two information is useful to know how much time your
request spends going from client to the server and you can optimize them time
usually working on the proximity between these two applications.</p>
<p>More in general you can parse a trace to get what ever you need as normal logs
or events the powerful things is downsampling and analysis.
If you are tracing a queue system you can get the average time for a worker to
process a message.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Let me know if you have more questions on twitter
<a href="https://twitter.com/gianarb">@gianarb</a>. I am happy to answer them here.</p>
Logs, metrics and traces are equally uselessThe key monitoring a distributed system is not logs, metrics or traces but how you are able to aggregate them. You can not observe and monitor a complex system looking at single signals.https://gianarb.it/img/dna.jpg2018-06-18T10:38:27+00:002018-06-18T10:38:27+00:00https://gianarb.it/blog/logs-metrics-traces-aggregation<p>Every signal from applications or infrastructure is useless, in the distributed
system era aggregation matters.</p>
<p>The ability to combine logs, metrics and, traces together is the key takeaway
here.</p>
<p>Kubernetes spin ups too many containers to allow us to stream or tail a log
fail.</p>
<p>Even cloud providers offer too many virtual machines to enable us to tail
logs.</p>
<p>A centralized place where to store all of them is a great start, but you
need to experience and learn how to combine the metrics you are ingesting to
increase the visibility over your system.</p>
<p>If you instrument your code with
opentracing, for example, you can get the <code>trace_id</code> and attach it to your log
to associate it with the trace itself. It can also work as the lookup key for
troubleshooting.</p>
<p>If you get some weird logs, you will know from where it comes.
With opentracing, this is still a bit of a mess the specification recently
<a href="https://github.com/opentracing/specification/blob/master/rfc/trace_identifiers.md">added explicit support to extract TraceId and SpanId from the
SpanContext</a>.
It is currently not implemented in a lot of implementation. I recently started a
conversation in the
<a href="https://github.com/opentracing/opentracing-go/issues/188">opentracing-go</a>
project to figure out how to apply it because currently it depends from what
tracer you are using and it is an essential regression for the specification
itself that should hide it by design.</p>
<p>Using Jaeger this is the way to do it:</p>
<pre><code>if sc, ok := span.Context().(jaeger.SpanContext); ok {
sc.TraceID()
}
</code></pre>
<p>Using Zipkin:</p>
<pre><code>zipkinSpan, ok := sp.Context().(zipkin.SpanContext)
if ok == true && zipkinSpan.TraceID.Empty() == false {
w.Header().Add("X-Trace-ID", zipkinSpan.TraceID.ToHex())
}
</code></pre>
<p>To get back in track, I wrote this article because I saw this problem and this
inclination speaking with friends, colleagues and other devs, we are now good
(or just better) storing high cardinality values but save them inside a database
doesn’t give us any value it is all about how we use them.</p>
<p>Correlation brings your alert to a different level. You probably have an alarm
to measure how much disks you still have.</p>
<p>An alert on the only CPU usage can be very frustrated
even more if it happens too often and a lot of time you restart a container or a
node to make it work because at 2 am you can’t fix the cause. You can
investigate what matters to fill an issue on GitHub.</p>
<p>Every automation tools can make your work leaving you free to sleep. It can
probably fill out the issue.</p>
<p>Combining the CPU with the time for the system to recover from a node restart
can make your alert smart enough to wake you up when it is not able to fix
itself leaving you ready for more acute and trivial problems.</p>
<h2 id="conclusion">Conclusion</h2>
<p>It is a pretty straightforward concept, but yes, everything is useless if you
store data without getting values out of them doesn’t matter if they are logs,
metrics or traces. The real value is not in a single one of them, it is in how
do you aggregate them together because a complex simple doesn’t explain itself
over one signal.</p>
Cloud Native Intranet with Kubernetes, CoreDNS and OpenVPNDesigning an architecture the network should be a top priority because it is very hard to change moving forward. Even in a cloud environment running on Kubernetes the situation doesn't change. Security and networking are hard pattern hard to inject in old projects. In this talk I will share a practical idea about how to start in the best way with OpenVPN and private DNS in a Kubernetes cluster in order to build your own intranet.https://gianarb.it/img/kubernetes.png2018-05-29T10:38:27+00:002018-05-29T10:38:27+00:00https://gianarb.it/blog/cloud-native-intranet<p>This article has a marketing and buzzword oriented title. I know.</p>
<p>Let me introduce you to what I am going to speak with you here with better
worlds: VPN, private, DNS, kubernetes, security.</p>
<p>I hope we all agree that VPN should be a must-have when you set up an
infrastructure. It doesn’t matter what you are doing, how many people are
working with you.</p>
<p>When you design a new system usually you need to expose to the public only some
service over HTTP and HTTPS all the rest: Jenkins, monitoring tools,
dashboards, log management should be locked-down and accessible just in a
private network. An intranet.</p>
<blockquote>
<p>An intranet is a private network accessible only to an organization’s staff.
Often, a wide range of information and services are available on an
organization’s internal intranet that is unavailable to the public, unlike the
Internet.</p>
</blockquote>
<p>All these concepts apply to “Cloud Native” ecosystem as well.</p>
<p>Kubernetes has a powerful dashboard and CTL that you can use to interact with
the API. That API doesn’t need to be publicly exposed, and to use the CLI from
your laptop, you should set up a VPN.</p>
<h2 id="openvpn">OpenVPN</h2>
<p>Usually, I configure an OpenVPN using the image
<a href="https://hub.docker.com/r/kylemanna/openvpn/">kylemanna/openvpn</a> available on
Docker Hub. It is straightforward to apply, and it offers a set of utilities
around user creation and certification management.</p>
<pre><code class="language-yml">apiVersion: v1
kind: Service
metadata:
name: openvpn
namespace: openvpn
labels:
app: openvpn
spec:
ports:
- name: openvpn
nodePort: 1194
port: 1194
protocol: UDP
targetPort: 1194
selector:
app: openvpn
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: openvpn
namespace: "openvpn"
labels:
app: openvpn
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: openvpn
template:
metadata:
labels:
app: openvpn
spec:
nodeSelector:
role: vpn
containers:
- name: openvpn
image: docker.io/kylemanna/openvpn
command: ["/etc/openvpn/setup/configure.sh"]
env:
- name: VPN_HOSTNAME
valueFrom:
configMapKeyRef:
name: vpn-hostname
key: hostname
- name: VPN_DNS
valueFrom:
configMapKeyRef:
name: vpn-hostname
key: dns
ports:
- containerPort: 1194
name: openvpn
securityContext:
capabilities:
add:
- NET_ADMIN
volumeMounts:
- mountPath: /etc/openvpn/setup
name: openvpn
readOnly: false
- mountPath: /etc/openvpn/certs
name: certs
readOnly: false
volumes:
- name: openvpn
configMap:
name: openvpn
defaultMode: 0755
- name: certs
persistentVolumeClaim:
claimName: openvpncerts
</code></pre>
<p>I put the persistentVolumeClaim to remember you to store in a persisted and
safe place (and you should backup them too) the certificates used and generated
by the VPN <code>/etc/openvpn/certs</code>.</p>
<p>I won’t write more about this topic; we are all excellent yaml developers!</p>
<p>How to create users, configuration and so on is a well knows topic that you can
easily <a href="https://openvpn.net/index.php/open-source/documentation.html">find in the OpenVPN’s
documentation</a>.</p>
<p>I don’t know if you realized that, but this VPN runs inside a Kubernetes
Cluster, so well configurated allow us to reach pods via a private network and
a bonus point also via kubedns to ping services, pods and all the other
resources registered to it.</p>
<p>To do that OpenVPN server can be configured to push kubedns to the client:</p>
<pre><code>dhcp-option DNS <kube-dns-ip>
</code></pre>
<p>Something with learned is that if you are using Linux the
NetworkManager-OpenVPN plugin pushes the DNS correctly, but the OpenVPN cli
tool doesn’t if you are using the last one you need to set it up in another
way.</p>
<p>Tips: You can take the <code><kube-dns-ip></code> doing <code>cat /etc/resolv.conf</code> from inside a pod.</p>
<h2 id="dns">DNS</h2>
<p>Push the KubeDNS or the DNS used by kubernetes is not enough to have a complete
intranet. You should be able to set up a custom domain to have friendly or
short URL.</p>
<p>You can take two different directions. KubeDNS can have static
record configured, but some person is not happy to touch or customize too much
the KubeDNS because Kubernetes itself use it and if you mess it up all it can
be a problem.</p>
<p>A possible solution is to deploy another DNS like CoreDNS and
configures it to resolve KubeDNS as a fallback. In this way, you will be free
to register custom LTDs and records. Kubernetes is going to use KubeDNS as
usual, and if you mess up CoreDNS, only a fraction of your system will blow
out.</p>
<p>Naturally to resolve your custom domains from the VPN you need to push
the CoreDNS ip and not the one used by Kubernetes.</p>
<p>If two DNSs are too much take the option one or from Kubernetes 1.10 you can
use CoreDNS as kubernetes DNS so it is a bit more flexible and you can use only
that one if you are brave enough.</p>
<p>I suggested CoreDNS because it supports records configuration via
<a href="https://github.com/coredns/coredns/tree/master/plugin/etcd">etcd</a>. Here an
example of Corefile:</p>
<pre><code>. {
errors
etcd *.myinternal {
stubzones
path /skydns
entrypoint http://etcd-1:2379,http://etcd-2:2379,http://etcd-3:2379
upstream /etc/resolv.conf
}
proxy . /etc/resolv.conf
}
</code></pre>
<p>Running this configuration inside a pod automatically fallback to kubedns (that
automatically fallback to the one configured to reach internet). Because of
<code>upstream</code> point to <code>resolv.conf</code> that inside a pod contains kubedns.</p>
<h2 id="benefits">Benefits</h2>
<p>Resolve Kubernetes DNS record from your local environment is very comfortable
to build a shared or dynamic development environment for you and your
colleagues.</p>
<p>You can set up per-developer namespaces that they can use to
deploy services reachable from the program that they are writing. Or you can
deploy your application, and another person connected to the VPN will be able
to use it.</p>
Server time vs Response timeHow do you dimension an infrastructure? How can you calculate container limits or how many nodes your application requires to support a specific load? Response time and server time are two key measurements to monitor saturation.https://gianarb.it/img/pastrami-sf.jpg2018-05-18T10:38:27+00:002018-05-18T10:38:27+00:00https://gianarb.it/blog/server-time-vs-response-time<p>If you find yourself in San Francisco walking nearby Market Street, you should
consider stopping at the Jewish Museum. There is a charming Pastrami place just
next to it. It is a sandwich place with good lemonade. It only takes 3-4 minutes
to get your meal, and from there it takes no more than 15 minutes walking to be
in front of the Ocean. Very nice!
Now, let’s consider this other scenario.
It is lunchtime, and you are starving. You rush outside your office, and you run
to the Pastrami place close to the Jewish Museum. After 35 minutes of wait, you
get your sandwich and start eating it asking yourself: why it took so long this
time? Shall I probably have walked to the next place to get a faster meal?</p>
<p>Something similar can happen to your Services as well! And that’s precisely the
phenomena in computer science we try to capture using the concepts of server
time and response time.
Server time aims to measures how much a server takes to run a specific action.
Let’s say consider an example operation the generation of a monthly report: it
usually takes 2ms, but if a lot of customers require the same kind of report at
the same time and your system saturates? This situation might very quickly end
up in having a subset of them getting the report in more than 1 minute or
actually in the timeout of the operation. The time it takes for a customer to
get his report is what is typically called response time.</p>
<h2 id="how-can-we-measure-these-metrics">How can we measure these metrics?</h2>
<p>The answer to this question is not easy: it depends on your architecture and
system. The starting point is instrumenting your application to determine how
much time it gets to produce the report. Stress testing is the other important
aspect: generating some load on your application and sampling the average
response time will let you estimate the application’s service time. Notice that
to make this measurement the app should NOT soak during this test!</p>
<p>If you control all the chain (from the HTTP app that sends the request to the
server), you can trace the request and simulate the same behavior of your
customers. If you can’t do this, you can consider using the frontend edge,
probably a load balancer.</p>
<blockquote>
<p>I would rather have questions that can’t be answered than answers that can’t
be questioned. Richard Feynman</p>
</blockquote>
<h2 id="why-does-it-matter">Why does it matter</h2>
<p>How many nodes do I need to deploy to accommodate x number of requests per
second? When should I consider scaling out my application? How does scale-out
affect the customer experience?
This is precisely why server time and response time matters! Having an average
response time close to the defined service time is a signal of proper
utilization and health of an application because it indicates that the response
latency is under control and it is far from saturation. Bringing to the limit
these two signals, in addition, is a key metric to estimate the correct sizing
of the applications instances and infrastructure.</p>
<p><img alt="Market Street San Francisco, Pastrami Resturant Jewish Meseum" src="/img/pastrami-sf.jpg" class="img-fluid" /></p>
<p>Btw the Pastrami place exists! You should try it! I will be in SF in 2 weeks. So
let me know about other places <a href="https://twitter.com/gianarb">@gianarb</a>.
Picture from GMaps. I will take a better one!</p>
Go how to cleanup HTTP request terminated.Cleaning up HTTP request, the most expensive one can be a huge performance improvement for your application. This short article shows how to handle HTTP request termination in Go.https://gianarb.it/img/gopher.png2018-04-25T10:38:27+00:002018-04-25T10:38:27+00:00https://gianarb.it/blog/go-http-cleanup-http-connection-terminated<p>Expensive HTTP handler is everywhere, doesn’t matter how good you are as a
developer. Business logic is what matters in our application, and it can be
pretty complicated. It can create large files, resources on AWS starts
thousands of containers on Kubernetes.</p>
<p>This kind of procedures have in common they can be very slow and they produce a
lot of garbage if the system/person who requires that stops prematurely by
mistake or not.</p>
<p>If your API requests create AWS resources and the client, terminate the call
you should clean what you created.</p>
<p>if you are generating a report and the customer changes are mind and refresh
you should stop the procedure.</p>
<p>You bet! Queues, background processes probably fit better but coming back on
the previous example, if you are computing something and who is waiting for the
result changed his mind, stop and release resources can be a massive
optimization.</p>
<pre><code class="language-bash">package main
import (
"fmt"
"io/ioutil"
"net/http"
"os"
"time"
)
func main() {
http.HandleFunc("/a", func(w http.ResponseWriter, r *http.Request) {
err := ioutil.WriteFile(os.TempDir()+"/txt", []byte("hello"), 0644)
if err != nil {
panic(err)
}
println("new file " + os.TempDir() + "/txt")
notify := w.(http.CloseNotifier).CloseNotify()
go func() {
<-notify
println("The client closed the connection prematurely. Cleaning up.")
os.Remove(os.TempDir() + "/txt")
}()
time.Sleep(4 * time.Second)
fmt.Fprintln(w, "File persisted.")
})
http.ListenAndServe(":8080", nil)
}
</code></pre>
<p>When you are building an HTTP server in Go, you can use a channel provided by
the Zhttp.ResponseWriter` to wait for the connection to be closed. And if it
happens, you can take action. The prototype above is very simple, every
request stores a file but I would like, remove the file if the client closes
the connection.</p>
<pre><code class="language-bash">$ run main.go
</code></pre>
<p>You can start the server, and from another terminal, you can start a <code>curl</code>, you
will see that after almost 4 seconds your request will succeed and the file
will be persisted on disk. Check it!</p>
<pre><code>$ time curl http://localhost:8080/a
File persisted.
real 0m4.018s
user 0m0.008s
sys 0m0.006s
$ cat /tmp/txt
</code></pre>
<p>Now let’s suppose that the client terminates the connection because it is too
slow or the person who made the request doesn’t care anymore.
Are you going to leave that request going? Event if nobody cares and it is just
consuming resources?</p>
<p>As you can see I am using the Notifier to remove the file if the client
terminates the connection:</p>
<pre><code class="language-go">notify := w.(http.CloseNotifier).CloseNotify()
go func() {
<-notify
println("The client closed the connection prematurely. Cleaning up.")
os.Remove(os.TempDir() + "/txt")
}()
</code></pre>
<p>You can check it stopping a <code>curl</code> just after starting it:</p>
<pre><code>$ time curl http://localhost:8080/a
^C
real 0m1.016s
user 0m0.008s
sys 0m0.005s
</code></pre>
<p>And the server reports</p>
<pre><code>$ go run main.go
new file /tmp/txt
The client closed the connection prematurely. Cleaning up.
</code></pre>
<p>That’s it! Build and clean after yourself!</p>
Go testing tricksThis post contains some feedback about how to write tests in Go.https://gianarb.it/img/gopher.png2018-04-17T10:38:27+00:002018-04-17T10:38:27+00:00https://gianarb.it/blog/go-testing-tricks<p>I recently wrote a blog post with my <a href="/blog/testing-shit">point of view about
testing</a>. I used Go as the language to concretize it. I had
good feedback about that article, and this is all about how I write tests in
Go.</p>
<h2 id="fixtures">Fixtures</h2>
<p>I wrote that I don’t like them, but I think they are useful. You can use them
to verify the same function checking the same assertion with different input.
So let’s say you are testing a function that returns the multiplication of two
numbers if the first is even, it returns the division if not.</p>
<p>I will write two tests, one to test events number and one to test the other
case, and I will set up two fixtures one for every test. I won’t write just one
test with elaborate fixtures because they are hard to read and the name of the
test function will help a lot to understand the assertion. Small example good
for blogging purpose. But I hope you got the idea.</p>
<pre><code class="language-golang">package test
import "testing"
func MagicFunction(f int, s int) int {
if f%2 == 0 {
return f * s
}
return f / s
}
func TestEventInputsShouldReturnMoltiplication(t *testing.T) {
table := []struct {
first int
second int
result int
}{
{2, 1, 2},
{4, 10, 40},
}
for _, s := range table {
if r := MagicFunction(s.first, s.second); r != s.result {
t.Errorf("Got %d, expected %d. They should be the same.", r, s.result)
}
}
}
func TestOddInputsShouldReturnDivision(t *testing.T) {
table := []struct {
first int
second int
result int
}{
{15, 3, 5},
{21, 7, 3},
}
for _, s := range table {
if r := MagicFunction(s.first, s.second); r != s.result {
t.Errorf("Got %d, expected %d. They should be the same.", r, s.result)
}
}
}
</code></pre>
<h2 id="sub-test">sub-test</h2>
<p>To make the fixtures a bit better I use the <code>t.Run</code> function a lot. It is a
feature introduced in Go 1.9 as part of the <code>testing</code> package.</p>
<pre><code class="language-go">package test
import (
"fmt"
"testing"
)
func MagicFunction(f int, s int) int {
if f%2 == 0 {
return f * s
}
return f / s
}
func TestEventInputsShouldReturnMoltiplication(t *testing.T) {
table := []struct {
first int
second int
result int
}{
{2, 1, 2},
{4, 10, 40},
}
for _, s := range table {
t.Run(fmt.Sprintf("%d * %d", s.first, s.second), func(t *testing.T) {
if r := MagicFunction(s.first, s.second); r != s.result {
t.Errorf("Got %d, expected %d. They should be the same.", r, s.result)
}
})
}
}
</code></pre>
<p><code>vim-go</code> has an option <code>let g:go_test_show_name=1</code> to allow the name of the
test as part of the output for :GoTest or :GoTestFunc. This helps a lot to
enjoy this feature.</p>
<h2 id="golden-files">Golden files</h2>
<p>Golden files are something used in different packages in the Go standard
library, and Michael Hashimoto spoke about it during his brilliant talk about
testing at the <a href="https://www.youtube.com/watch?v=8hQG7QlcLBk">GopherCon 2017</a>.
In case of complex output, you can verify the result of the tests with the
content of a file. It improves order and readability. When you declare a
global flag in your test, it becomes available inside <code>go test</code> so if you use
the update flags all the tests will pass, but you will update all the golden
files. So this is very useful if you need to compare a lot of bytes.</p>
<pre><code class="language-go">update := flag.Bool("update-golden-files", false, "Update golden files.")
</code></pre>
<pre><code class="language-sh">go test -update-golden-files
</code></pre>
<p>I was using this trick a lot when I was writing PHP code, and I was testing HTTP responses.</p>
<h2 id="test-helper-and-return-function">Test helper and return function</h2>
<p>When you have repeatable code across tests, you can create a helper function,
and you can use it in your tests. There are two general rules about this
approach:</p>
<ol>
<li>The helper function should have access to *t testing.T variable.</li>
<li>Your helper never returns an error; it marks the test as failed. That’s why
it needs access to <code>*t. testing.T</code>.</li>
</ol>
<p>Another good trick is to return a function from the helper to clean up what you
did in the helper. So let’s say that your helper starts an HTTP server. You can
return the HTTP Close function as a callback.</p>
<pre><code class="language-go">func () testHelperStartHTTPServer(t *testing.T) func() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// long and complex mock maybe with a golden file and so on
}))
return func() { ts.Close() }
}
func TestYourTest(t *testing.T) {
hclose := testhelperStartHTTPServer()
// All your logic and checks
defer hclose()
}
</code></pre>
<p>I used the same practice when I was writing integration tests using bash and
<a href="https://github.com/sstephenson/bats">bats</a>. It is a very clean and easy to
read approach.</p>
<h2 id="parallel">parallel</h2>
<p>You can use the function <code>t.Parallel()</code> to notify at the test runner that your
case can run in parallel with other tests marked as parallel. When you write
unit tests, you can almost always run them in parallel because they should be
completely isolated.</p>
<h2 id="short-and-verbose">Short and verbose</h2>
<p><code>-short</code> and <code>-v</code> are two flags available when you run <code>go test</code>. You can use
them in your tests:</p>
<pre><code>import "testing"
func TestVeryLongAndExpensiveCapability(t *testing.T) {
if testing.Short() {
t.Skip("skipping this testsVeryLongAndExpensive is too expensive")
}
// ... other code
}
</code></pre>
<p><code>-short</code> describes itself pretty well, you can skip tests that are too long and expensive.</p>
<p><code>-v</code> allows you to print more:</p>
<pre><code>import "testing"
func TestVeryLongAndExpensiveCapability(t *testing.T) {
if testing.Verbose() {
}
// ... other code
}
</code></pre>
<h2 id="testingquick">testing/quick</h2>
<p><a href="https://golang.org/pkg/testing/quick/">testing/quick</a> is a nice package that
offers a set of utilities to write test quick. Go has not an assertion library
inside the stdlib but this can help if you are like me and you are happy to not
vendor assertion libraries because <code>if { }</code> with some sugar is what I need.</p>
<p>So that’s it, have fun and write tests!</p>
The Go awesomenessAfter 1 year writing go every day at work this is why I like to work with it.https://gianarb.it/img/fight-club.jpg2018-04-09T10:38:27+00:002018-04-09T10:38:27+00:00https://gianarb.it/blog/go-awesomeness<p>It’s one year since I started to use Go every day at work. I was using it before
but for fun or OSS projects. I was looking for my next challenge, and I was
mainly working with PHP, JavaScript previously and I knew that a compiled,
statically typed language was my next step.</p>
<p>At my previous job at CurrencyFair, the environment was pretty standard for a
financial tech company so backend in Java, frontend in PHP. But my experience
with all the interfaces and abstract classes that I created in Java at that time
made me hate that language. So I was looking for something different.</p>
<p>I was as I am now involved in automation, cloud and
operational other than development so all the tools like Docker, InfluxDB,
Kubernetes, Consul, Vault was in golang and for me as OSS addicted it become the
natural choice.
Now after all this time I am ready to write why I think Go is the right choice
for me now.</p>
<h2 id="1-abstraction-and-maintainability">1. abstraction and maintainability</h2>
<p>I wrote a lot about <a href="/blog/the-abstract-manifesto">this topic</a>, so I am not going to repeat myself. But I
think maintainability is tied together with abstraction. Previously when I was
working with PHP, we always had services, injection and so on. In that
environment it was good, but all that abstraction like in Java doesn’t make your
code more flexible. It makes it hard to understand in the long run and code
needs to be written with history in mind because delete code is very hard.
Go with its interface implementation, how it forces you to struct the project
helps the codebase to grow in a better way.</p>
<h2 id="2-stdlib">2. Stdlib</h2>
<p>Community wasted time across languages to identify the right way to indent code.
Go comes with that decision done. Same for testing. How to write automatic
tests, benchmarks is inside the language. No libraries, it is there.
More in general os, net, net.http, img and so on, a lot of stuff are provided by
the language itself. It is great because you don’t need anything to start,
other than Go. Compared with other languages you can do a lot more things.
Having all this feature inside Go guarantees compatibility over time, they won’t
break compatibility for the next years, and the code is developed and reviewed
by a large number of people.</p>
<h2 id="3-pprof">3. pprof</h2>
<p>pprof is a profiler, and it is shipped as part of Go. You can use it even via
cli, or it also has an excellent HTTP package under
<a href="https://golang.org/pkg/net/http/pprof/">net/HTTP/pprof</a>.
Just to show you how much power it can be InfluxDB extends it to export a zip
archive with all the information we need to troubleshoot the database behavior:</p>
<pre><code class="language-go">func (h *Handler) handleProfiles(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/debug/pprof/cmdline":
httppprof.Cmdline(w, r)
case "/debug/pprof/profile":
httppprof.Profile(w, r)
case "/debug/pprof/symbol":
httppprof.Symbol(w, r)
case "/debug/pprof/all":
h.archiveProfilesAndQueries(w, r)
default:
httppprof.Index(w, r)
}
}
</code></pre>
<p>Here all the code
<a href="https://github.com/influxdata/influxdb/blob/442581d299b7d642e073bbe42112fa9b58fb071a/services/httpd/pprof.go#L21">influxdata/influxdb</a>.
This is super useful because we can ask customers or developer in the OSS
community to export and upload the archive to see what is going on.
Having a standard way to troubleshoot and export a profile allows us to build
visualization or static analysis on it for common calculation.</p>
<h2 id="4-delve">4. delve</h2>
<p>A good debugging session is the best way to approach a new application or to go
deeper learning a language or a software.
<a href="https://github.com/derekparker/delve">delve</a> is easy to setup and to use. Even
if you are not gdb/debugger superhero as I am not you will be able to make your
first steps with delve. So it is a nice starting point too.</p>
<h2 id="5-godoc">5. godoc</h2>
<p>Other than behind an excellent way to generate documentation from source code I
use it a lot even when I am not designed libraries just to double check that my
package has the comprehensive public methods. I always think about what I am
exposing to the outside when I write code. APIs are not just JSON or HTTP thing,
every object exposes their API, and you need to be aware of how you are building
iteration between the internal state and the outside. Avoid misuse of your
structs is your responsibility as developer and godoc help me to identify poor
decision.</p>
<h2 id="6-vim-go">6. vim-go</h2>
<p>I would like to stay in my terminal all day, and vim-go allows me to write good
code in my comfort zone. In the past I wrote a lot of vim scripts and plugins,
following how fatih and all the other maintainers are developing
<a href="https://github.com/fatih/vim-go">vim-go</a> is great.
Bonus point they recently added support for delve, so you can now debug golang
application in vim!</p>
<h2 id="7-dep">7. dep</h2>
<p>Dependency management is probably the worst things that Go has. The good thing
is that now we have [dep] and it should become the standard way to manage
dependencies. Right now the situation looks a lot like this:</p>
<p><img class="img-fluid" src="/img/fight-club.jpg" /></p>
<p>Govender, go get, glide currently there are a lot of different ways to manage
dependencies, and it generates a lot of confusion, but I hope at the end we will
converge in just one. Probably dep.</p>
<h2 id="conclusion">Conclusion</h2>
<p>More in general with Go I am learning that the language is one of expect to
become a good developer. A good developer needs to know the language, but the
best way to go deeper in it is writing tests, benchmarks, profiling application
and using the debugger. All these tools make my life as developer easy. Easy
life for me means that I can go deeper solving problems and indirectly it will
make me a better developer.</p>
<p>Go is fun!</p>
Observability according to mePrometheus, InfluxDB, TimescaleDB, Cassandra and all the time series databases that show up every week is a clear sign that now we need more than just a way to store metrics. I think is now clear that collecting more metrics is the point. More data is not directly related to a deeper understanding of our system.https://gianarb.it/img/mountain-garbage.jpg2018-04-04T10:38:27+00:002018-04-04T10:38:27+00:00https://gianarb.it/blog/observability<p>I started to read about observability almost one year ago, <a href="https://twitter.com/mipsytipsy">Charity
Major</a>
comes to my mind when I thinking about this topic and she is the person that is
pushing very hard on this.</p>
<p>This is probably the natural evolution of how we approach monitoring.</p>
<p>Distributed systems require a different way to approach the three monitoring
piles: collect, storage and analytics.</p>
<p>Understand a microservices environment brings a new layer of complexity and the
most obvious consequence is the amount of data that we are storing compared, I
think it is way more than before.</p>
<p>Prometheus, InfluxDB, TimescaleDB, Cassandra and all the time series databases
that show up every week is a clear sign that now we need more than just a way to
store metrics. I think is now clear that collecting more metrics is the point.
More data is not directly related to a deeper understanding of our system.</p>
<p>Observability for a lot of companies look like a new way to sell analytics
platform but according to me it’s a scream to bring us back to the problem: “How
can we understand what is happening?” or even better “How should we use the data
we have to understand what’s going on?”.</p>
<p>All the data should be organized, reliable and usable. Logs, metrics, traces
are part of the resolution, the brain to analyze and get value out from them is
what Observability means to me.</p>
<p>Visualization is one expect, proactive monitoring, correlation, and hierarchy
are other steps. Looking at our old graphs all of them are driven by the
hostname for example. But now we have containers, we have virtual machines and
immutable infrastructure makes rebuilding less costly and more secure than an
incremental update. The name of the server should not be the keyword for our
queries, the focus should be moved to the role of services.</p>
<p>Think about your Kubernetes cluster, you label servers based on what they will
run, if something unusual happens the first things to do is to move the node out
from the production pool, the autoscaler will replace it and you will be
troubleshoot it later.</p>
<p>Before we were looking at processes, we were keeping them alive as the Olympic
flame but containers are making them volatile. We spin them up and down for
every request in some serverless environment. What we care and what we should
monitor are the events that float across our services, that’s the new gold. W we
can lose 1000 containers but we can’t miss the purchase order made my a
customer. All our effort should be moved on that side.</p>
<p>I love this point of view because it brings us to what really matters, our
applications.</p>
<p><img src="/img/mountain-garbage.jpg" class="img-fluid" /></p>
<p>According to me the mountain of waste showed in the picture explains really well
our current situation, we collected what ended up to be a lot of garbage and now
we need to climb it looking for a better point of view. I think the data in our
time series databases are not garbage but gold, it’s just not simple as it
should be.</p>
<p>That’s why is great that companies are building tools to fill the gap:
<a href="https://github.com/influxdata/ifql">IFQL</a> is an example. The idea behind the
project is to build a language to query and manipulate data in an easy way. Same
for company line Honeycomb or open source projects like Grafana and Chronograf
that are trying to make these data easy to use.</p>
<p>We spoke about tools but there is another big expect and it’s all a cultural,
distributed teams need different tools to collaborate and troubleshoot problems.
Different UI and way to interact with graph and metrics.</p>
I don't give a shit about testingThis is all about how I approach testing in development. TDD, DDD, unit test, integration test. It should make my development faster and my code easy to maintain. We have a lot of different techniques because we need to be good on picking the right one.https://gianarb.it/img/laziness.jpg2018-03-29T10:38:27+00:002018-03-29T10:38:27+00:00https://gianarb.it/blog/testing-shit<p>That’s what I learned during my experience as a developer. Doesn’t matter
which languages you end up working if you are making HTTP API or things like
that you don’t have an excuse. Write tests make your development faster, and it will
drastically improve maintainability of your project.</p>
<p>In this post, I would like to tell you how I approach testing particularly in
Go obviously.</p>
<p>First of all, when you create a new file, you should write its <code>_test.go</code> child.
It’s hard to tell you who should be the child of who. Sometimes I start
to write everything inside a test function, just because run the actual test is faster
compared with compile, run the binary, trigger the right entry point and so on.
When I am satisfied, I move the
code to a function, and I leave the assertions I wrote as a new test. <strong>Pretty good</strong>.</p>
<blockquote>
<p>I don’t give a shit about automated testing. I write tests.</p>
</blockquote>
<p>I use <a href="https://github.com/fatih/vim-go"><code>vim-go</code></a> and <code>:GoTestFunc</code> is probably
the most used shortcut during my day to day job.</p>
<p>When I can choose I don’t use assertion libraries, the <code>testing</code> package is
enough for me and dependency management in go is a pain, so fewer things I vendor
better I feel about myself.</p>
<p><img src="/img/laziness.jpg" class="img-fluid" /></p>
<p>I use fixtures, but I don’t like them. I prefer to write more small tests than complicated fixtures.</p>
<p>A single test for me is more descriptive, and I don’t mind to write redundant code, I can always refactor it later or move it some helper function. A complicated fixture will be hard to maintain.
The name of the function is an excellent way to describe what you are covering in your test and the function itself
creates a beautiful block that improves readability.</p>
<pre><code class="language-go">func TestCarComposition(t *testing.T) {
fixtures := []car.Composition{
{"blue", "europe", 1, nil, "2011-12-05", "ford"},
{"", "", 35, bool, nil, "ford"},
{"red", "usa", 0, bool, nil, "fiat"},
{"white", "", 35, bool, nil, "kia"},
{"orange", "", 1, bool, "2010-05-12", nil},
{"", "", 0, bool, nil, "ford"},
}
}
</code></pre>
<p>Bonus point, as you can see fixtures are sad to read!</p>
<p>Even unit vs. integration vs. function is a very annoying discussion. Don’t tell me
about TDD, BDD, CCC, DDD things. I don’t care they are all amazing as
soon as they can make my development simple.</p>
<p>So, CDD is probably my best test methodology: <strong>Comfort driven development</strong>.</p>
<p>Usually when I am writing a computing function when it elaborates maps, strings,
files without using too many external resources I start from unit test, because
it makes iteration faster as I told before. And it won’t require too many mocks.
I don’t like mocks.</p>
<h2 id="lets-discuss-mocks">Let’s discuss mocks</h2>
<p>Mocks are a pain; you end up to be bored when you write mocks, they won’t fail when it’s useful for you to see an error and they will fail when you don’t care.
So comfort looks very far from mocks!</p>
<p>When mocks becomes too complicated, and I can write another kind of tests I go with that solution. Maybe integration or I will try to write the simple mock
possible, sometimes even the entire web server can be a valuable solution:</p>
<pre><code class="language-go">func TestInfluxDBSdkGetTheRightValues(ti *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
data := influxdb.Response{
Results: []influxdb.Result{
{
Series: []influxdbModels.Row{},
},
},
}
w.Header().Add("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_ = json.NewEncoder(w).Encode(data)
}))
defer ts.Close()
config := influxdb.HTTPConfig{Addr: ts.URL}
client, _ := influxdb.NewHTTPClient(config)
defer client.Close()
// Whatever you need to check
}
</code></pre>
<p>You need to play carefully; these tests are slower and more expensive in
resources.
But I like the idea to take the faster solution when I am developing; you can
come back on your tests later when the feature is more stable and better
designed. Write tests should not slow me down too much, I am looking for a way
to write the implementation and the test fasts to iterate on both of them other than waste time making everything perfect. Nothing will be forever; nothing will be complete in programming, so design your environment to be easy to change.</p>
<h2 id="integration-tests">Integration tests</h2>
<p>I am a CLI kind of person, so I often send HTTP requests via cURL.
Docker is very easy from day one to start and stop your application,
cleaning databases and so on.</p>
<p><a href="https://github.com/sstephenson/bats"><code>bats</code></a> combines these two sentences. It is an automation test framework for
bash. It is very simple to setup, and it allows me to copy paste some cURL, and
with jq, you can make the checks you need over your JSON response.</p>
<p>An integration test suite made with bats looks like that:</p>
<ol>
<li>An “init” file in bash where you can run setup and teardown function before and after every test. Usually, you can you that functions to spin up and down the
containers that you need to tests, this is the one that I wrote for this
example</li>
</ol>
<pre><code class="language-bash">#!/bin/bash
function setup() {
teardownCallback=$(init)
}
function teardown() {
eval $teardownCallback
}
function getHost() {
echo "http://localhost"
}
function init {
executionID=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 7 | head -n 1)
containerLabels="exec=${executionID}"
$(docker run -d -l $containerLabels -p 80:80 nginx)
echo "docker ps -aq -f 'label=${containerLabels}' | xargs docker rm -f"
}
</code></pre>
<ol>
<li>You have a set of <code>.bats</code> files with the various scenarios, I wrote one to
check if the status code 200 for the nginx home</li>
</ol>
<pre><code class="language-bash">#!/usr/bin/env bats
load utils
@test "Nginx home return 200" {
statusCode=$(curl -I -X GET "$(getHost)" 2>/dev/null | head -n 1 | cut
-d$' ' -f2)
[ $statusCode -eq 200 ]
}
</code></pre>
<p>What you are running is a <code>bats</code> test to check that <code>nginx:latest</code> is serving the right page.
Your use case will be ten times more complicated.</p>
<p>Another reason to take this approach is about bash itself. If you are not a bash
expert, you will probably end up to write straightforward tests, cURL, grep,
regex and some pipes. Nothing more.</p>
<p>And you won’t use any code that runs your application. It’s important to avoid
weird buggy tests.</p>
<h2 id="developer-happiness">developer happiness</h2>
<p>Tests are a methodology to decrease the cost of maintenance and to improve your
ability to write code.</p>
<p>It should not be a fashion way to show how good you are as a developer. You will
be a good developer as a side effect.</p>
<p>I look at all the different way to tests my code as a tool set, AI is becoming
very smart. So we need to be less “server” and more human been. 100% coverage
for unit tests looks a lot like something that a server can do. Pick the right
method based on your feeling.</p>
<script>
$(document).ready(function() {
$('body').css("background", "#F5F3E6");
});
</script>
Review book Database Reliability EngineerReview Book Database Reliability Engineer. Author Laine Campbell and Chairty Major. Published by O'Reillyhttps://gianarb.it/img/dbre-book.jpg2018-03-27T09:08:27+00:002018-03-27T09:08:27+00:00https://gianarb.it/blog/database-reliability-engineer-review<p>The author of <a href="https://amzn.to/2FE5z4V">Database Reliability Engineer</a> are Laine Campbell, Charity Majors.
It is published by O’Reilly. You can buy it on Amazon.</p>
<p>You probably know the book <a href="https://gianarb.it/blog/site-reliability-engineering-review">Site Reliability
Engineer</a>, if you don’t I reviewed
it a few days ago.</p>
<p><img src="/img/dbre-book.jpg" class="img-fluid" /></p>
<p>This book walks the same experience but focused on databases. I work for
InfluxData as SRE and I deal every days with databases running on our Cloud
Product so I am in that topic but now because I am good as DBA but just as a
developer with a background on distributed system and cloud computing.</p>
<p>It doesn’t really matter if you manage databases as DBA or if you use it as a
developer this book contains useful contents for both categories.</p>
<p>It is a dance book, I started it ready it few months ago and at some point I
spotted, just because it contains too many notions and experiences and it
requires some time to metabolize them.</p>
<p>I particularly enjoyed the Chapter 7 about <strong>Backup and Recovery</strong>. But also Chapter
3 about about <strong>Risk Management</strong> is great because it goes deeper on how metrics
should drive your way to look at risks and outage.</p>
<p>My daily job is all around database orchestration, containers and so on. So I
found this book very useful for my day to day job and it enforced the expectation
that I had about the application that I am building.</p>
<p>I will try to go deeper on backup management to make the recovery part of a more
structure pipeline to be sure that it is always usable for example.</p>
<p>It is a very practical book based and you can fill all the enthusiasm and the
experiences made by the two authors Charity and Laine. If you are happy to
learn from who has its hands dirty this book is your book. It drives you an
story and less learned.</p>
<p>How to manage migration and how to fill the gap between developer and DBA
because they are both the same goals the success of the company and the project.</p>
<p>If we can keep both on the same loop avoiding backstabbing we will increase our
chance of success. If you are a manages and you see some friction in your teams,
this book can give you some good feedback.</p>
How to use a Forwarding Proxy with golangCloud, Docker, Kubernetes make your environment extremely dynamic, it has a lot of advantages but it adds another layer of complexity. This article is about forward proxy and golang. How to configure your http Client to use an http, https forward proxy for your golang application to increase security, scalability and to have a set of public ips for outbound traffic.https://gianarb.it/img/gopher.png2018-03-21T09:38:27+00:002018-03-21T09:38:27+00:00https://gianarb.it/blog/golang-forwarding-proxy<p>A forwarding proxy is a proxy configuration that handle requests from a set of
internal clients that are trying to create a connection to the outside.</p>
<p>In practice is a man in the middle between your application and the server that you are
trying to connect. It works over the HTTP(S) protocol and it is implemented at the
edge of your infrastructure.</p>
<p>Usually, you can find it in large organizations or universities and it is used as
additional control mechanism for authorization and security.</p>
<p>I find it useful when you work with containers or in a dynamic cloud environment
because you will have a set of servers for all the outbound network
communication.</p>
<p>If you work in a dynamic environment as AWS, Azure and so on you will end up
having a variable number of servers and also a dynamic number of public IPs.
Same if your application runs on a Kubernetes cluster. Your container can be
everywhere.</p>
<p>Now let’s suppose that a customer asks you to provide a range of public IPs
because he needs to set up a firewall… How can
you provide this feature? In some environments can be very simple, in others
very complicated.</p>
<p>1st December 2015 a users asked this question on the <a href="https://discuss.circleci.com/t/circleci-source-ip/1202">CircleCI
forum</a> this request is
still open. This is just an example, CircleCi is great. I am not complaining
about them.</p>
<p>One of the possible ways to fix this problem is via the forwarding proxy. You can
spin up a set of nodes with a static ip and you can offer the list to the
customer.</p>
<p>Almost all cloud providers have a way to do that, floating ip
on DigitalOcean or Elastic IP on AWS.</p>
<p>You can configure your applications to forward the requests to that pool and
the end services will get the ip from the forward proxy nodes and not from the
internal one.</p>
<p>This can be another security layer for your infrastructure because you will be
able to control and scan packages that are going outside from your network in a
really simple way and in a centralized place.</p>
<p>This is not a single point of failure because you can spin up more than one
forward proxies and they scale really well.</p>
<p>Under the hood, a forward proxy is the <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT">HTTP method
<code>CONNECT</code></a>.</p>
<blockquote>
<p>The CONNECT method converts the request connection to a transparent TCP/IP
tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an
unencrypted HTTP proxy.</p>
</blockquote>
<p>A lot of HTTP Client across languages already support this in a very
transparent way. I built a very small example using golang and
<a href="https://www.privoxy.org/">privoxy</a> to show you how simple it is.</p>
<p>First of all, let’s build an application called <code>whoyare</code>. It is an HTTP server
that returns your remote address:</p>
<pre><code class="language-go">package main
import (
"encoding/json"
"net/http"
)
func main() {
http.HandleFunc("/whoyare", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
body, _ := json.Marshal(map[string]string{
"addr": r.RemoteAddr,
})
w.Write(body)
})
http.ListenAndServe(":8080", nil)
}
</code></pre>
<p>You can call the <code>GET</code> the route <code>/whoyare</code> and you will receive a JSON like
<code>{"addr": "34.35.23.54"}</code> where <code>34.35.23.54</code> is your public address. Running
<code>whoyare</code> from your laptop if you make a request on your terminal you should get
<code>localhost</code> as remote address. You can use curl to try it:</p>
<pre><code class="language-bash">18:36 $ curl -v http://localhost:8080/whoyare
* TCP_NODELAY set
> GET /whoyare HTTP/1.1
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 18 Mar 2018 17:36:40 GMT
< Content-Length: 31
<
* Connection #0 to host localhost left intact
{"addr":"localhost:38606"}
</code></pre>
<p>I wrote another application, it uses <code>http.Client</code> to print the response on
stdout. If you have the server running you can run it:</p>
<pre><code class="language-go">package main
import (
"io/ioutil"
"log"
"net/http"
"os"
)
type whoiam struct {
Addr string
}
func main() {
url := "http://localhost:8080"
if "" != os.Getenv("URL") {
url = os.Getenv("URL")
}
log.Printf("Target %s.", url)
resp, err := http.Get(url + "/whoyare")
if err != nil {
log.Fatal(err.Error())
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err.Error())
}
println("You are " + string(body))
}
</code></pre>
<p>So, this is a very simple example, but you can apply this example to more
complex environments.</p>
<p>To make this example a bit more clear I created two virtual machines on
DigitalOcean, one will run privoxy the other one will run <code>whoyare</code>.</p>
<ul>
<li><strong>whoyare</strong>: public ip 188.166.17.88</li>
<li><strong>privoxy</strong>: public ip 167.99.41.79</li>
</ul>
<p>Privoxy is a very simple to setup forward proxy, nginx, haproxy doesn’t fit very
well for this use case because they do not support the CONNECT method.</p>
<p>I built a docker image
<a href="https://hub.docker.com/r/gianarb/privoxy/"><code>gianarb/privoxy</code></a>, it’s on Docker
Hub. You can run and by default, it runs on port 8118.</p>
<pre><code class="language-bash">core@coreos-s-1vcpu-1gb-ams3-01 ~ $ docker run -it --rm -p 8118:8118
gianarb/privoxy:latest
2018-03-18 17:28:05.589 7fbbf41dab88 Info: Privoxy version 3.0.26
2018-03-18 17:28:05.589 7fbbf41dab88 Info: Program name: privoxy
2018-03-18 17:28:05.591 7fbbf41dab88 Info: Loading filter file:
/etc/privoxy/default.filter
2018-03-18 17:28:05.599 7fbbf41dab88 Info: Loading filter file:
/etc/privoxy/user.filter
2018-03-18 17:28:05.599 7fbbf41dab88 Info: Loading actions file:
/etc/privoxy/match-all.action
2018-03-18 17:28:05.600 7fbbf41dab88 Info: Loading actions file:
/etc/privoxy/default.action
2018-03-18 17:28:05.607 7fbbf41dab88 Info: Loading actions file:
/etc/privoxy/user.action
2018-03-18 17:28:05.611 7fbbf41dab88 Info: Listening on port 8118 on IP address
0.0.0.0
</code></pre>
<p>The second step is to build and scp <code>whoyare</code> in your server. You can
build it using the command:</p>
<pre><code>$ CGO_ENABLED=0 GOOS=linux go build -o bin/server_linux -a ./whoyare
</code></pre>
<p>Now that we have the application up and running we can try via cURL to query it
directly and via privoxy.</p>
<p>Let’s try directly as we did previously:</p>
<pre><code>$ curl -v http://your-ip:8080/whoyare
</code></pre>
<p><code>cURL</code> uses an environment variable <code>http_proxy</code> to forward the requests through
the proxy:</p>
<pre><code>$ http_proxy=http://167.99.41.79:8118 curl -v http://188.166.17.88:8080/whoyare
* Trying 167.99.41.79...
* TCP_NODELAY set
* Connected to 167.99.41.79 (167.99.41.79) port 8118 (#0)
> GET http://188.166.17.88:8080/whoyare HTTP/1.1
> Host: 188.166.17.88:8080
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 18 Mar 2018 17:37:02 GMT
< Content-Length: 29
< Proxy-Connection: keep-alive
<
* Connection #0 to host 167.99.41.79 left intact
{"addr":"167.99.41.79:58920"}
</code></pre>
<p>As you can see I have set <code>http_proxy=http://167.99.41.79:8118</code> and the response
doesn’t contain my public ip but the proxy one.</p>
<p><img src="/img/frankenstain-jr.jpg" alt="" /></p>
<p>These are the logs that you should expect from privoxy for the requests crossing it:</p>
<pre><code>2018-03-18 17:28:22.886 7fbbf41d5ae8 Request: 188.166.17.88:8080/whoyare
2018-03-18 17:32:29.495 7fbbf41d5ae8 Request: 188.166.17.88:8080/whoyare
</code></pre>
<p>The client that you run it previously by default it connects to <code>localhost:8080</code>
but you can override the target URL via env var <code>URL=http://188.166.17.88:8080</code>.
Running the following command I reached directly <code>whoyare</code>.</p>
<pre><code>$ URL=http://188.166.17.88:8080 ./bin/client_linux
2018/03/18 18:37:59 Target http://188.166.17.88:8080.
You are {"addr":"95.248.202.252:38620"}
</code></pre>
<p>The golang <code>HTTP.Client</code> supports a set of environment
variables to configure the proxy, it makes everything very flexible because
passing
these variables to any service already running it will just work.</p>
<pre><code>export HTTP_PROXY=http://http_proxy:port/
export HTTPS_PROXY=http://https_proxy:port/
export NO_PROXY=127.0.0.1, localhost
</code></pre>
<p>The first two are very simple, one is the proxy for the HTTP requests, the
second for HTTPS. <code>NO_PROXY</code> excludes a set of hostname, the hostname listed
there won’t cross the proxy. In my case localhost and 127.0.0.1.</p>
<pre><code>HTT_PROXY=http://forwardproxy:8118
+--------------+ +----------------+ +----------------+
| | | | | |
| client +----------^+ forward proxy +--------^+ whoyare |
| | | | | |
+--------------+ +----------------+ +----^-----------+
|
|
+---------------+ |
| | |
| client +-------------------------------------------+
| |
+---------------+
HTTP_PROXY not configured
</code></pre>
<p>The client with the environment variables configured will cross the forward
proxy. Other client will reach it directly.</p>
<p>This granularity is very important. It’s very flexible because other than a
“per-process” you can also select what request to forward and what to exclude.</p>
<pre><code>$ HTTP_PROXY=http://167.99.41.79:8118 URL=http://188.166.17.88:8080
./bin/client_linux
2018/03/18 18:39:18 Target http://188.166.17.88:8080.
You are {"addr":"167.99.41.79:58922"}
</code></pre>
<p>As you can see we just reached <code>whoyare</code> via proxy and the <code>addr</code> in response is
now ours but the proxy one.</p>
<p>The last command is a bit weird but it is just to show how the <code>NO_PROXY</code> works.
We are calling the proxy excluding the <code>whoyare</code> URL, and as expected it doesn’t
cross the proxy:</p>
<pre><code>$ HTTP_PROXY=http://167.99.41.79:8118 URL=http://188.166.17.88:8080 NO_PROXY=188.166.17.88 ./bin/client_linux
2018/03/18 18:42:03 Target http://188.166.17.88:8080.
You are {"addr":"95.248.202.252:38712"}
</code></pre>
<p>Let’s read this article as a practical introduction to golang, forward proxy. You can
subscribe to my <a href="/atom.xml">rss feed</a> or you can follow me on
<a href="https://twitter.com/gianarb">@twitter</a>. Probably I will write about how to
replace <code>privoxy</code> with golang and about how to setup and deploy this solution on
Kubernetes. So let me know what to write first!</p>
The abstract manifestoOften looking at the code I spot a lot of places where it looks too complicated. Disappointment is the feeling that I get reading classes with weird names or chain of abstractions or interfaces used only one time. Abstraction is often the reason for all my sadness.https://gianarb.it/img/shit-pretty.png2018-03-17T10:08:27+00:002018-03-17T10:08:27+00:00https://gianarb.it/blog/the-abstract-manifesto<p>This is a personal outburst. Stop to abstract by default.</p>
<p>I worked on too many application abstracted by default. And abstracted I mean
complicated.</p>
<p>The abstraction is easy to understand when you need it. If you need to think too
much about why that crappy code should have an interface, or that made should
implement an interface and not an object you are out of track.</p>
<p>Abstraction is not the answer, code architecture is, unit testing helps,
integration tests are the key to the modern microservices environment.</p>
<p>Don’t waste your time creating interfaces that nothing will reuse. If you don’t
know what to do run.</p>
<p>There are languages and design pattern that probably set your brain to look for
abstraction everywhere. I worked with Java developer that wasn’t able to write a
class without an interface, or without its abstract. My question was: “Why are
we doing that?”. Compliance.</p>
<blockquote>
<p>Dude, your world is a very boring one, and you are the root cause.</p>
</blockquote>
<p>If you are working in a service-oriented environment with services big enough
to be rewritten easily the abstraction is even more useless.</p>
<p>We are developers, we often don’t build rockets. That’s life, there are a good
amount of companies that make rockets, apply there or you will put your company
in the condition of paying technical debts for you and they will hire smart
contractors to figure out what you did now that you are not working there
because after probably just one year you locked yourself in that boring project
full of complicated concepts.</p>
<p>btw, I don’t think that software to control rockets has a lot of abstractions
sorry.</p>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Y'all are all about passionate programmers, but honestly I'd
rather programmers than care _just enough_. I could do with less pedantic
arguments about code.</p>— 。 𝕷𝖎𝖓𝖉𝖘𝖊𝖞 𝕭𝖎𝖊𝖉𝖆 。 (@lindseybieda) <a href="https://twitter.com/lindseybieda/status/969296749985779712?ref_src=twsrc%5Etfw">March
1, 2018</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>I saw this tweet on my timeline yesterday and I think it really describes my
current mood. The code changes over time and I should spend more time making it
flexible enough to support this continues to grow. Abstraction is not the right
way.</p>
<p>So, passionate code engineer always abstract? That’s not the giveaway, Java
engineer always abstract? maybe.</p>
Review book Site Reliability EngineeringA review about Site Reliability Engineering a book published by O'Reilly about Google and it's massive scale from the point of view of the engineers that made that scale possible. Distributed system, microservices and data driven development.https://gianarb.it/img/sre-book.jpeg2018-03-15T10:08:27+00:002018-03-15T10:08:27+00:00https://gianarb.it/blog/site-reliability-engineering-review<p>I bought <a href="https://amzn.to/2pfeHBU">Site Reliability Engineering</a> a lot of months
ago. I read the ebook first but I am the kind of people that buy also paper
books when they are good, so if you are working on a distributed and scalable
environment it’s something that you should read.</p>
<p>Published by O’Reilly and edited by Betsy Beyer, Chris Jones, Jennifer Petoff
and Niall Richard Murphy it is written by many Google engineers and it’s about
the experience they made scaling services like Google Maps, Calendar, YouTube
and all the other products.</p>
<p>I spoke with different people about this book and a lot of them told me that
there is nothing new on that. It’s just cool because Google made it cool.</p>
<p>I have a different option. It’s a nice book because it is a complete source of
information about design and processes in a highly scalable environment.
Probably
some of them topic are well known but it’s hard to find all this information in
a single place.</p>
<p><img alt="Site Reliability Engineering book" src="/img/sre-book.jpeg" class="img-fluid" /></p>
<p>To be fair, it has 524 pages so it’s not a fast read. It took me few
months but I keep it around when I need to explain concepts like how to
dimension and measure loads in a services environment. SLA, SLO and how to use
them properly to manage and measure risks are
well explained, circuit breaking and more, in general, a lot of good procedures
about
resiliency, teamwork, delivery are explained in this book.</p>
<p>There is a nice chapter about how to use the metrics to set up a function and
smart alerting system to keep engineer on-call in a safe and comfortable
environment.</p>
<p>Another one about how Google design resilient applications and how they
dimension services. How much and how deep they know their services impressed me
a lot.</p>
<p><strong>Site Reliability Engineering</strong> is a good mix of concepts that you can apply
through your day to day no-at-Google job and the all the
Google scale “freaky fun”.</p>
<p>So, in the end, I will define like the bible for an engineer that wish to work
in a high-scalable environment. Doesn’t matter if you are not there yet or if
you
won’t serve millions of requests per second. It’s good to read and to keep
around.</p>
<p>The <a href="https://landing.google.com/sre/book/index.html">HTML version</a> of this book
is now available online for free.</p>
What is distributed tracing. Zoom on opencensus and opentracingDistributed tracing is a fast growing concept. We increased the distributions for our applications and the consequence is a different complexity to monitor and understand what is going on across regions and applications (microservices). With this article I share something about what tracing is and my experience with opentracing and opencensus.https://gianarb.it/img/gianarb.png2018-02-18T10:08:27+00:002018-02-18T10:08:27+00:00https://gianarb.it/blog/what-is-distributed-tracing-opentracing-opencensus<p>A few months ago I started to actively study, support and use opentracing and
more in general the distributed tracing topic.</p>
<p>In this article, I will share something about what I have learned starting from
the basic I hope to get your thoughts and questions via Twitter
<a href="https://twitter.com/gianarb">@gianarb</a>.</p>
<p>We all know the trend for the last couple of years. Spread applications across
containers, cloud providers and split them into smallest units called services
or microservices, pets…</p>
<p>This procedure brings a lot of advantages:</p>
<ul>
<li>you can manage people in a better way and spread them across this small units.</li>
<li>small units are easy to understand for new people or after a couple of months.
In a work like our where there is a high turnover having the ability to
rewrite a service if nobody knows it in a couple of days it’s great.</li>
<li>You can monitor these units in a better way and if you detect scalability
problems or bottleneck you can stay focused on the specific problem without
having other functions around. It enforces the single responsibility in some
way.</li>
</ul>
<p>Btw there are other points for sure, but the last one is very important and I
think it helps us to understand why tracing is so important now.</p>
<p>We discover that monitor this pets is very hard and it’s different compared to
the previous situation. A lot of teams discovered this complexity moving forward
with services making noise in production.</p>
<p>Our focus is not on the virtual machine, on the hostname or the even on the
container. I don’t care about the status of the server. I care about the status
of the service and even deeper I care about the status of a single event in my
system. This is also one of the reasons why tools like Kafka are so powerful and
popular. Reply a section of your history and collect events like user
registration, new invoice, new attendees register at your event, new flight
booked or new article published are the most interesting part here.</p>
<p>Servers, containers should be replaceable things and they shouldn’t be a
problem. The core here is the event. And you need to be 100% sure about having
it in someplace.</p>
<p>Same for monitoring, if the servers, containers are not important but events are
you should monitor the event and not the server.</p>
<p>Oh, don’t forget about distribution. It makes everything worst and more
complicated my dear. Events move faster than everything else. They are across
services, containers, data centers.</p>
<p>Where is the event? Where it failed. How a spike for particular events behave on
your system? If you have too many new registrations are you still able to serve
your applications?</p>
<p>In a big distributed environments what a particular service is calling? Or is it
used? Maybe nobody is using it anymore. These questions need to have an answer.</p>
<p>Distributed tracing is one of the ways. It doesn’t solve all the problems but it
provides a new point of view.</p>
<p>In practice speaking in HTTP terms tracings translate on following a specific
request from its start (mobile app, web app, cronjobs, other apps) so it’s the .</p>
<p>Registering how many applications it crosses to, for how long. Labeling these
metrics you can event understand latency between services.</p>
<p><img src="https://www.hawkular.org/img/blog/2017/2017-04-19-jaeger-trace.png" class="img-fluid" />
<small>from https://www.hawkular.org/</small></p>
<p>Speaking in the right language, this image describes a trace. It’s an HTTP to
<code>fronted</code> service. It’s a GET request on the <code>/dispatch</code> route. You can see how
far you can go. A trace is a collection of spans.</p>
<p>Every span has it’s own id and an optional parent id to create the hierarchy.
Spans support what is called Span Tags. It is a key-value store where the key is
always a string and some of them are “reserved” to describe specific behaviors.
You can look at them <a href="https://github.com/opentracing/specification/blob/master/semantic_conventions.md#standard-span-tags-and-log-fields">inside the specification
itselt</a>.
Usually, UI is using this standard tag to build a nice visualization. For
example if a span contains the tag <code>error</code> a lot of tracers colored it red.</p>
<p>I suggest you read at the standard tags because it will give you the idea about
how descriptive a span can be.</p>
<p>The architecture looks like this:</p>
<pre><code> +-------------+ +---------+ +----------+ +------------+
| Application | | Library | | OSS | | RPC/IPC |
| Code | | Code | | Services | | Frameworks |
+-------------+ +---------+ +----------+ +------------+
| | | |
| | | |
v v v v
+-----------------------------------------------------+
| · · · · · · · · · · OpenTracing · · · · · · · · · · |
+-----------------------------------------------------+
| | | |
| | | |
v v v v
+-----------+ +-------------+ +-------------+ +-----------+
| Tracing | | Logging | | Metrics | | Tracing |
| System A | | Framework B | | Framework C | | System D |
+-----------+ +-------------+ +-------------+ +-----------+
</code></pre>
<p><small>from <a href="https://opentracing.io/documentation/pages/instrumentation/common-use-cases.html" target="_blank">opentracing.org</a><small></small></small></p>
<p>There are different instrumentation libraries across multiple languages and you
need to embed one of them in your application. It usually provides a global
variable where you can add spans too. Time by time they are stored in the
tracer that you select. If you are using Zipkin as tracer you can select
different backends like ElasticSearch and Cassandra.
Tracers provides API and UI to store and visualize traces.</p>
<p>As you can see from the graph above Opentracing “is able” to push to Tracers,
Logging system, metrics and so on. With my experience with opentracing, I don’t
know how this can be done.</p>
<p>I always used it with a Tracer like Zipkin or Jaeger to store spans. Logs are
covered by the spec because you can attach to every spans one or multiple <code>Span
Logs</code>.</p>
<blockquote>
<p>each of which is itself a key:value map paired with a timestamp. The keys must
be strings, though the values may be of any type. Not all OpenTracing
implementations must support every value type.</p>
</blockquote>
<p><small>from <a href="https://github.com/opentracing/specification/blob/master/specification.md" target="_blank">opentracing.org</a><small></small></small></p>
<p>The idea behind this feature is clear. There are too many buzzwords: metrics,
logs, events, time series and now traces.</p>
<p>It’s easy to end up with more
instrumentation libraries that business code. That’s probably why opentracing
cover this uses case. Logs and traces are time series. That’s probably why
metrics are there.</p>
<p>Using the go-sdk it looks like this:</p>
<pre><code class="language-go">span, ctx := opentracing.StartSpanFromContext(ctx, "operation_name")
defer span.Finish()
span.LogFields(
log.String("event", "soft error"),
log.String("type", "cache timeout"),
log.Int("waited.millis", 1500))
</code></pre>
<p>But I am not able to find a way to say: “Forward all these logs to ….elastic
and this traces to Zipkin”. And I don’t know if the expectation is to have
tracers smart enough to do that. But from my experience trying to extend Zipkin,
this looks like a hard idea. At first, because the tracers are out of the
OpenTracing’s scope.</p>
<p>If the goal is to wrap together everything logs have precise use case from ages.
They work pretty well and you can’t change the expectation. They can be a
real-time stream on stdout, stderr and/or other thousands of exporter. I can’t
find this kind of work there. So, looking at the code it’s not clear who is in
charge of what. But the graph is pretty.</p>
<p>I like the idea and I started looking at <a href="https://opencensus.io/">OpenCensus</a> a
library open sourced by Google from its experience with StackDriver and the
Google’s scale. It has its
<a href="https://github.com/census-instrumentation/opencensus-specs">specification</a> and
it provides a set of <a href="https://github.com/census-instrumentation/">libraries</a>
that you can add to your application to get what they call stats, traces out
from your app. Stat stays for metrics, events. It’s another buzz probably!</p>
<p>The concept looks similar to OpenTracing, obviously, the specs are different.</p>
<p>Looking at the code, the go-SDK looks a lot more clear. I can clearly see stats
and tracing objects, they both accept exporters and they can be Prometheus,
Zipkin, Jaeger, StackDriver and so on. I like the idea that the exporter is part
of the project, you don’t need a tracing application like Zipkin, you can write
your exporter to store data in your custom database and you are ready to go.</p>
<pre><code>.
├── appveyor.yml
├── exporter
│ ├── jaeger
│ ├── prometheus
│ ├── stackdriver
│ └── zipkin
├── internal
├── plugin
├── README.md
├── stats
│ ├── internal
│ ├── ...
│ └── view
├── tag
├── trace
</code></pre>
<p>You can probably do the same with OpenTracing writing your tracer that store
things in your custom databases jumping Zipkin and Jaeger, it looks a bit more
complicated looking at the interface:</p>
<pre><code class="language-go">// opencensus-go/trace/export.go
// Exporter is a type for functions that receive sampled trace spans.
//
// The ExportSpan method should be safe for concurrent use and should return
// quickly; if an Exporter takes a significant amount of time to process a
// SpanData, that work should be done on another goroutine.
//
// The SpanData should not be modified, but a pointer to it can be kept.
type Exporter interface {
ExportSpan(s *SpanData)
}
</code></pre>
<pre><code>// opentracing tracer
type Tracer interface {
// Create, start, and return a new Span with the given `operationName` and
// incorporate the given StartSpanOption `opts`. (Note that `opts` borrows
// from the "functional options" pattern, per
// https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis)
//
// A Span with no SpanReference options (e.g., opentracing.ChildOf() or
// opentracing.FollowsFrom()) becomes the root of its own trace.
//
StartSpan(operationName string, opts ...StartSpanOption) Span
// Inject() takes the `sm` SpanContext instance and injects it for
// propagation within `carrier`. The actual type of `carrier` depends on
// the value of `format`.
//
// OpenTracing defines a common set of `format` values (see BuiltinFormat),
// and each has an expected carrier type.
//
// Other packages may declare their own `format` values, much like the keys
// used by `context.Context` (see
// https://godoc.org/golang.org/x/net/context#WithValue).
//
Inject(sm SpanContext, format interface{}, carrier interface{}) error
// Extract() returns a SpanContext instance given `format` and `carrier`.
//
// OpenTracing defines a common set of `format` values (see BuiltinFormat),
// and each has an expected carrier type.
//
// Other packages may declare their own `format` values, much like the keys
// used by `context.Context` (see
// https://godoc.org/golang.org/x/net/context#WithValue).
//
Extract(format interface{}, carrier interface{}) (SpanContext, error)
}
</code></pre>
<p>OpenTracing doesn’t care about exporter and tracers, something else handle that
complexity, (the user, me.. bored) the standard only offers interfaces. I don’t
know if this is good. It really looks a lot more like a common interface
between traces. I like the idea, but I need a lot more.</p>
<p>Now, writing this article I understood that I have a lot more to figure out
about this projects, sadly I realized that in practice they are even more
similar compared my feeling before writing all this down.</p>
<p>Tracing, metrics and instrumentation libraries remain crucial from my point of
view. You can write everything you want but if you are not able to understand
what’s happening you are not making a good job. You look like a monkey.</p>
<p>Personal I would like to find a common and good library to wrap together all
the buzzwords stats, spans, traces, metrics, time series, logs because they are
all the same concept just from a different point of view.</p>
<p>Everything is a point in time, grouped, ordered or with a specific hierarchy.
You can use them as aggregate, to compare, to alert and so on. A powerful
implementation should be able to combine both needs an easy ingestion with a
clear output.</p>
<p>I think that OpenTracing has a lot to do from both sides in and out. OpenCensus
looks good from an ingestion point of view. Nothing about logs in OpenCensus
maybe because they are good enough as they are but we need to be able to cross
reference logs, traces, metrics from infrastructure and application events from
dashboards and automatic tools.</p>
<p>It looks like, with both setup, that you still need a platform capable to serve
and use this data. A lot of people will answer that it’s out of scope for
these projects, but I am pretty sure we all learned that just storing events is
not enough.</p>
The right balanceMy daily job as a developer is to find the right balance about everything. I would like to share what I think about this topic because the decisions that you take writing a system are the result of the software itself. So you should care about them.https://gianarb.it/img/k8s-up-and-running.jpg2018-02-09T10:08:27+00:002018-02-09T10:08:27+00:00https://gianarb.it/blog/the-development-balance<p>My daily job as a developer other than programming is about finding the right
balance between different things:</p>
<ol>
<li>Buzzword driven development vs rock solid boring things.</li>
<li>New technologies vs something already adopted inside the company.</li>
<li>State of the art implementation. Something that can work in an hour.</li>
</ol>
<p>And many more. I am sure you will have your own list. What I noticed during
these years working for different companies, with different teams and in
different states is that there are a lot of kind of developers, we are full of
companies and projects to develop.</p>
<p>You should really look for the right place for you, but you need to know what to
look for.</p>
<p>What I am trying to say that people, colleagues, companies can help you to find
your balance be proactive on this research. Speak with your manager or with your
colleagues about what you are happy to do or not you will find interesting
answers if you are working with people with your same mindset. You will probably
come up selling and buying tasks and issues from member of the teams because
they like more what you are doing and vice versa.</p>
<p>Sometime you will even have the sensibility to grab a bored task just to have it
done and leave your colleagues free to do something more fun.</p>
<p>This is the kind of work that I like. I am almost sure about that now. A place
where shit needs to be done and you have an active word on how, it doesn’t need
to be a decision, sometime I don’t know what’s better for the company or for the
project, but can’t be always a black box that come from the ground and needs to
be done. I am not an check list implementer and dealing with my colleagues and
other people is an active and very good part of my day.</p>
<p>When it’s time to have fun because things are going well and even when you are
the unique different voice out form a meeting. Find the right place where you
feel free to say your opinion, be wise and mature enough to know that your
opinion can be good or bad can’t be always the one to follow.</p>
<p>The right balance between all of these things, across teams, companies gives to
me the feel to be in the right place.</p>
Kubernetes up and runningKubernetes up and running review.https://gianarb.it/img/k8s-up-and-running.jpg2017-12-19T10:08:27+00:002017-12-19T10:08:27+00:00https://gianarb.it/blog/kubernetes-up-and-running<p>I read <a href="https://amzn.to/2zflChj">“Kubernetes up and running”</a> an O’Reilly book
written by Kelsey Hightower, Brendan Burns and Joe Beda.</p>
<p>It looks like the instrumental manual, you look for it when you buy something.
Based on your knowledge of it you read or not that manual.</p>
<p>I have a good knowledge of containers, orchestrator and cloud computing but I
never worked with Kubernetes until 2 weeks ago when I started a co-managed k8s
cluster on scaleway with <a href="https://twitter.com/fntlnz">Lorenzo</a>.</p>
<p>The book is well written, I read it in less than one week. The chapters are well
split because I was able to jump “Builiding a Raspberry Pi Kubernetes cluster”
because I am not really interested without any pain.</p>
<p>Chapters like “Deploying real world applications”, “Service Discovery” are good
and the book covers all the basic concepts that you need to know about
Kubernetes. You can feel all the experience that the three authors have on the
topic. There are gold feedbacks about what they learned using and building what
is now the orchestration standard.</p>
<p>Just to summarize, if you are using kubernetes and you like papers, this book is
a good way to have a documentation on paper. If you are new to Kubernetes is the
best way to start.</p>
<p>Thanks to all the authors! If you have any questions let me know
<a href="https://twitter.com/gianarb">@gianarb</a></p>
Desk setupIt is not a couple of months after i left CurrencyFair to start working at InfluxData. A lot of new things but working from home for a US based company is very hard. Dealing with a such big timezone requires a big effort. But I am very excited about how I am feeling working from home. That's why I decided to share my current office setup. Desktop, Zendbook and a lot of Ikea's things!https://gianarb.it/img/docker.png2017-12-17T10:08:27+00:002017-12-17T10:08:27+00:00https://gianarb.it/blog/desk-setup<p>As you probably know in April 2017 I moved back after an year and half in Dublin
and I started to work from home as SRE at InfluxData.</p>
<p>I am ready to write a small post about my current setup.</p>
<h2 id="ikea-markus">Ikea Markus</h2>
<p>First things I bought an <a href="https://www.ikea.com/gb/en/products/chairs-stools-benches/office-chairs/markus-swivel-chair-glose-black-art-20103101/">Ikea
Markus</a>.
It is comfortable and it has a competitive price. It’s flexible and well designed.</p>
<p>I don’t have a lot to say about it. If you are not passionate about expensive
and weird chairs you can go for this one. It will work!</p>
<h2 id="stand-mount">Stand Mount</h2>
<p>My setup counts two boring Asus monitor, one horizontal and one vertical,
Lorenzo suggested to me this <a href="https://amzn.to/2yMe59C">standmount support for two monitor</a>.</p>
<p>Day by day I discover how bad I am using more than one monitor. Change focus so
often is not for me but I like the vertical monitor when I am debugging some
weird application.</p>
<h2 id="asus-zenbook-3">Asus Zenbook 3</h2>
<p>I have an <a href="https://amzn.to/2AHAy9N">Asus Zenbook 3</a> the only usb-c is kind of a
pain. I am a traveler and a speaker. I don’t enjoy its low weight (900gr) that
often because I always need some adapter.</p>
<p>For traveling the adapter <a href="https://amzn.to/2CKIMPG">Asus Universal Dock</a> is good. It
is embedded with a charger it means that you need to have power supply to use
it. I wrote an article about it and I was very disappointed about the product.
But now that I am using it only for traveling purpose it’s not too bad.</p>
<p>If you are a multi desktop user you need to remember that it doesn’t have an
external video card, it has VGA and HDMI but you can use only one of them at the
time.</p>
<p>I used Ubuntu 17.04 and 17.10. Now I am using ArchLinux and both laptop and
Universal Dock need to install some drivers, to work a bit on audio
configuration and so on. But it’s a good challenge and almost everything works
out of the box.</p>
<h2 id="logitech-c922">Logitech C922</h2>
<p>The embedded Asus WebCam is not great. If you are looking for high definition or
an acceptable quality you need to have an external webcam.</p>
<p>I work from and when I have a meeting with colleagues and friends I would like
to offer to them good experience.</p>
<p>The <a href="https://amzn.to/2kEnJ9o">Logitech C922</a> is not powerful enough to make me
beautiful but it makes an amazing work and it’s very good.</p>
<p>If you record tutorials or workshop you should think about having one of this.
It comes with a small tripod to setup it where ever you like.</p>
<h2 id="usb-c-adapter">USB-C adapter</h2>
<p>As I told you the world is not ready for the USB-C but I am!
<a href="https://amzn.to/2zhPbSQ">Plugable</a> makes my life very simple.</p>
<p>WebCam, two monitors, Ethernet cable are always attached to it and I just need
to plug my laptop in via the USB-C and everything will happen.</p>
<p>It’s an expensive toy but I am using it on Linux and it’s working. The company
doesn’t officially support it but there is a
<a href="https://github.com/displaylink/evdi">DisplayLink</a> driver open source on GitHub
that you can use.</p>
<h2 id="desk">Desk</h2>
<p>Last but not least I have a standing desk.</p>
<p>I think a good chair, gym, swim are better solution to keep you healthy but I
change my point of view and my position help me to stay focused.</p>
<p>Every time I have a boring or complex task at some point just toggle my
current position from down to up or vice versa gives me some fresh power to
end it well.</p>
<p>I monitored the <a href="https://www.ikea.com/gb/en/products/desks/office-desks/bekant-desk-sit-stand-oak-veneer-black-spr-29061187/">Ikea
Bekant</a>
for a lot of months but I was not sure about investing money a standind desk.</p>
<p>I looked at them for so long that Ikea started a very good discount campaign and
I just bought it. I took only the mechanical legs because I like the feeling of
real wood and I bought the table separately.</p>
<p>That’s it! Bye!</p>
From Docker to MobyDocker announced during DockerCon a new project called Moby. Moby will be the new home for Docker and all the other open source projects like containerd, linuxkit, vpnkit and so on. Moby is the glue for all that open source code. It will look as an entire platform to ship, build and run containers at scale.https://gianarb.it/img/docker.png2017-10-20T10:08:27+00:002017-10-20T10:08:27+00:00https://gianarb.it/blog/from-docker-to-moby<p>At DockerCon 2017 Austin
<a href="https://blog.docker.com/2017/04/introducing-the-moby-project/">Moby</a> was the
big announcement.</p>
<p>It created confusion and some communities are still trying to understand what is
going on. I think it’s time to step back and see what we have after seven months
after the announcement.</p>
<ol>
<li><code>containerd</code> is living a new life, the first stable release will happen soon.
It has been donated to CNCF.</li>
<li><code>notary</code> is the project behind <code>docker trust</code>. I wrote a full e-book about
<a href="https://scaledocker.com">Docker Security</a> if you need to know more. This
also has been donated to the CNCF.</li>
<li>github.com/docker/docker doesn’t exist anymore there is a new repository
called github.com/moby/moby .</li>
<li><a href="https://github.com/docker/cli">CLI</a> has a separate home.</li>
<li>docker-ce is the first example of moby assembling. It is made my Docker Inc.</li>
</ol>
<p>Containers are not a first class citizen in Linux.</p>
<p><img class="img-fluid" src="/img/container-is-not-real.jpeg" /></p>
<p>They are a combination of cgroups, namespaces and other kernel features. They are
also there from a lot of year. LXD is one of the first project that mentioned
container but the API wasn’t really friendly and only few people are using it.</p>
<p>Docker created a clean and usable api that human beings are happy to use. It
created an ecosystem with an amazing and complete UX. Distribution, Dockerfile,
<code>docker run</code>, <code>docker image</code> and so on.</p>
<p>That’s what Docker is, in my opinion. Other than a great community and a fast
growing company.</p>
<p>What Docker is doing with Moby is to give the ability to competitors, startups, new
projects to join the ecosystem that we built in all these 4 years.</p>
<p>Moby in other hands is giving the ability at Docker to take ownership of the
clean and usable experience. The <code>Docker CLI</code> that we know and use every day
will stay open source, but not the moby project’s part. It will be owned by
Docker. As I wrote above, the code is already moved out.</p>
<p>Moby allows other companies and organisations to build their
user interface based on what they need. Or to build their product on top of a
open source project designed to be modular.</p>
<p>Cloud and container moves fast Amazon with ECS, RedHat with OpenShift,
Pivotal with Cloud Foundry, Mesos with Mesosphere, Microsoft with Azure
Container Service, Docker with Docker, they are all pushing hard to build
projects around containers to sell them at big and small corporations to make
legacy projects less bored.</p>
<blockquote>
<p>Legacy is the new buzzword</p>
</blockquote>
<p>Docker will continue to assemble and ship docker as we know it. The project is
called <code>docker-ce</code>:</p>
<pre><code>apt-get install docker-ce
docker run -p 80:80 nginx:latest
</code></pre>
<p>Everything happens down the street, in the open source ecosystem. Moby won’t
contain the CLI that we know.</p>
<p>Moby won’t have the swarmkit integration as we know it. It was something that
Docker as company was looking to have. Mainly to inject an orchestrator in
million of laptops. Other companies and projects that are not using swarm don’t
need it and they will be able to remove it in some way.</p>
<p>Companies like Pivotal, AWS are working on
<code>containerd</code> because other the runtime behind Docker it’s what matters for a lot
of projects that are just looking to run containers without all the layers on
top of it to make it friendly. ECS, Cloud Foundry are the actual layers on top
of “what runs a container”.</p>
<p>Container orchestrator doesn’t really care about how or who spins up a container,
they just need to know that there is something able to do that.</p>
<p>It is what Kubernetes does with CRI. They don’t care about Docker, CRI-O,
containerd. It’s out of scope they just need a common interface. In this case is
a gRPC interface that every runtime should implement. Here a list of them:</p>
<ul>
<li><a href="https://github.com/kubernetes-incubator/cri-o">cri-o</a></li>
<li><a href="https://github.com/kubernetes-incubator/cri-containerd">cri-containerd</a></li>
<li><a href="https://github.com/kubernetes-incubator/rktlet">rktlet</a></li>
</ul>
<p>That’s a subset of reasons about why everything is happening:</p>
<ul>
<li>Docker Inc. will be free to iterate on their business services and projects
without breaking every application in the world. And they will have more
flexibility on what they can do as company.</li>
<li>The transaction between Docker to Moby is the perfect chance to split the
project to different repositories we already spoke about docker-cli, containerd
and so on.</li>
<li>Separation of concern is popular design pattern. Split
projects on smallest libraries allow us to be focused on one specific scope of the
project at the time.
<a href="https://github.com/moby/buildkit">buildkit</a> is the perfect example. It’s the
evolution of the <code>docker build</code> command. We had a demo at the MobySummit and
it looks amazing!</li>
</ul>
<p>That’s almost it. Let’s summarise:</p>
<p><strong>Are you a company in the container movement?</strong>
You are competing with Docker building container things and you was complaining
about them breaking compatibility or things like that now you should blame the
Moby community.</p>
<p><strong>Are you using docker run?</strong>
You are fine! You will be able to do what you was doing before.</p>
<p><strong>Are you a OpenSource guru?</strong>
Maybe you will be a bit disappointed if you worked hard on docker-cli and now
Docker will bring your code out but you signed a CLA, the CLI will stay open
source. Blame yourself.</p>
<p>That’s it! Or at least that’s what I understood.</p>
Git git git, but betterDoesn't matter for how much time you are using git or any version control system, you always have something to learn abut them. Not about the actual interface but about the right mindset.https://gianarb.it/img/git.png2017-10-10T10:08:27+00:002017-10-10T10:08:27+00:00https://gianarb.it/blog/git-git-git-but-better<p>I can’t say that Git is a new topic. Find somebody unable to explain how a
version control system was working was very hard. Now it’s almost impossible.</p>
<p>I used SVN and Git for many years and I also put together some unusual use case
for example: <a href="https://devzone.zend.com/6134/splitting-zend-framework-using-the-cloud/">“Splitting Zend Framework Using the Cloud”</a>
is a project that I made with Corley SRL my previous company and the Zend
Framework team.</p>
<p>It helped me to put my hands down on the Git file system and I discovered a lot
of features and capabilities that are not the usual: commit, checkout, reset,
branch, cherry-pick, rebase and so on.</p>
<p>But during my experience building cloud at InfluxData I need to say that I can
see a change of my mindset, I am sharing this because I am kind of proud of
this. It’s probably not super good looking at the time required to achieve this
goals but how cares!</p>
<blockquote>
<p>Sometimes it’s the journey that teaches you a lot about your destination.
(Drake)</p>
</blockquote>
<p>I don’t know this Drake, I am not even sure if it’s the right author of the
quote but that’s not the point.</p>
<p>At InfluxData, just to give you more context, I am working on a sort of
scheduler that provisions and orchestrate servers and containers on our <a href="https://cloud.influxdata.com/">cloud
product</a>. A lot of CoreOS instances, go, Docker and
AWS api calls.</p>
<p>It’s a modest codebase in terms of size but it is keeping up a huge amount of
servers, I am actively working on the code base almost by myself and I am kind
of enjoying this. Nate, Goller and all the teams are supporting my approach and
are using it but I am not using Git because hundreds of developers need to
collaborate on the same line of code. I had some experience in that environment
working as contributor in many open source project. But this time is different.</p>
<p>I am mainly alone on a codebase that I didn’t start and I don’t know very well,
this project is running in production on a good amount of EC2.</p>
<p>I really love the idea of having a clean and readable Git history. I am not
saying that because it’s cool. I am saying that because every time I commit my
code I am thinking about which file to add/commit <code>-a</code> is not really an option
that I use that much anymore. I think about the title and the message.</p>
<p>I try to avoid the <code>WIP</code> message and I use it only if I am sure about a future
squash, rebase and if I need to push my code to ask for ideas and options (as I
said I am writing code almost alone, but I am always looking for support from my
great co-workers).</p>
<p>This has a very big value I think also as remote worker. This is my first
experience in this environment and for a no-native English a good and
self explanatory title can be the hardest part of the work but it will help
other people to understand what I am doing.</p>
<p>When you are working on a new codebase and you have tasks that require
refactoring to be achieved in a fancy and professional way you will find
yourself moving code around without really be able to figure out when and how it
will become useful to close your task and open the PR that your team lead is
waiting for. At the end if you start to write code and you commit your changes
at the end of the day as I was doing at the beginning after a couple of days you
will figure out that your PR is too big and you are scared to merge them.
And probably it’s just the PR that is preparing the codebase to get the initial
requests. I hated the situation but if you think about what I wrote you will
find that it’s totally wrong.</p>
<p>VCS is not there as saving point, you are not plaining Crash Bandicoot anymore,
you don’t need to use Git as your personal “ooga booga”. The right commit
contains an atomic information about a feature, bug fix or whatever.</p>
<p><img src="/img/crash_bandcioot.jpg" alt="" /></p>
<p>These are the questions that I am asking myself now before to make a commit:</p>
<ul>
<li>am I confident cherry-picking this commit to <code>master</code>? This is a good way to
make your commit small and easy to merge. If one of your PR is becoming too
big and you have “cherry-picked” commits you can select some of them merge
them as single PR.</li>
<li>are deploy and rollback easy actions? This is similar to previous one but I am
the one that deploy and monitor the service in production. I need to ask this
question to myself before every merged PR.</li>
<li>Looking at the name of the branch that in my case the task in my
viewfinder the commit that I am creating is about it or can I create a new PR
just for this piece of code? This helps me a log to split my PR and to them
small. A small PR is easier to review, it has a better scope and it makes me
less scared to deploy it.</li>
</ul>
<p>Git is more than a couple of commands that you can execute. You need to
be in the right mindset to enjoy all the power.</p>
Orbiter the Docker Swarm autoscaler on the road to BETA-1Orbiter is a project written in go. It is an autoscaler for Docker containers. In particular it works with Docker Swarm. It provides autoscaling capabilities for your services.https://gianarb.it/img/docker.png2017-08-09T10:08:27+00:002017-08-09T10:08:27+00:00https://gianarb.it/blog/orbiter-the-swarm-autoscaler-moves<p>Orbiter is an open source project written in go hosted on
<a href="https://github.com/gianarb/orbiter">GitHub</a>. It provides autoscaling
capabilities in your Docker Swarm Cluster.</p>
<p>As you probably know at the moment autoscaling is not a feature supported
natively by Docker Swarm but this is not a problem at all.</p>
<p>Docker Swarm provides a useful API that helps you improving its capabilities.</p>
<p>I created Orbiter months ago as use case with InfluxDB and to allow services to
scale automatically based on signal <code>up</code> or <code>down</code>. You can follow the webinar
that I made with InfluDB
<a href="https://www.influxdata.com/resources/influxdata-helps-docker-auto-scale-monitoring/?ao_campid=70137000000Jgw7">here</a>.</p>
<p>This article is not about “How it works”. You can <a href="https://gianarb.it/blog/orbiter-docker-swarm-autoscaler">read more here about how it
works</a> and you can
watch the embedded video that I made in the Docker HQ in San Francisco.</p>
<p>Yesterday we made some very good improvements and we are moving forward to tag
the first beta release. I need to say a big thanks to <a href="https://github.com/mbovo">Manuel
Bovo</a>. He coded pretty much all the features listed
here.</p>
<ol>
<li>
<p><a href="https://github.com/gianarb/orbiter/pull/26">PR #26</a> e2e working example. <a href="https://github.com/gianarb/orbiter/tree/master/contrib/swarm">Please try
it</a>.</p>
</li>
<li>
<p><a href="https://github.com/gianarb/orbiter/pull/27">PR #27</a> Now Orbiter has
background job that listen on the Docker Swarm event API and register and
de-register new services <a href="https://github.com/gianarb/orbiter#autodetect">deployed with right
labels</a>. You don’t need to
restart orbiter anymore. It detect new services automatically.</p>
</li>
<li>
<p><a href="https://github.com/gianarb/orbiter/pull/29">PR #29</a> Fixed the up/down range.
Now we can not scale under 1 tasks but we can scale up services with 0 tasks.</p>
</li>
<li>
<p><a href="https://github.com/gianarb/orbiter/pull/31">PR #31</a> We have a cooldown
period configurable via label <code>orbiter.cooldown</code>. This fix avoid multiple
scaling in a short amount of time.</p>
</li>
<li>
<p><a href="https://github.com/gianarb/orbiter/pull/32">PR #32</a> We are migrating our API
base root. Now all the API are <code>/v1/orbiter/.....</code>. At the moment we are
supporting old and new routes. <strong>In October I will remove the old one. Please
migrate to <code>/v1/orbiter/....</code> now!</strong>.</p>
</li>
</ol>
<h2 id="now">Now?</h2>
<p>That’s a good question, but I have part of the answer. In October the plan is to
release a BETA and finally the first stable version but what we need to do to go
there?</p>
<ul>
<li>Offer a proper auth method. Manuel started this
<a href="https://github.com/gianarb/orbiter/pull/33">PR</a>. I have some concerns but
we are on the right path.</li>
<li>Make orbiter “Only-Swarm”. The project started with the vision to become a
general purpose autoscaler. But this is not in line with the idea of single
responsibility and we designed a very clean API for Docker Swarm, make it
usable in other context is not going to work. We tried it with DigitalOcean
but the api and the project looks too complex and I love simplicity.</li>
<li>Get other feedback from the community to merge valuable features before the
stable release.</li>
</ul>
<p>That’s it! Share it and give it a try! For any question I am available on
twitter (@gianarb) or open an issue.</p>
Asus universal dock station driverEvery developer loves to speak about its setup. I am here to share my trouble with my new laptop. Asus Zenbook 3.https://gianarb.it/img/gianarb.png2017-08-03T10:08:27+00:002017-08-03T10:08:27+00:00https://gianarb.it/blog/asus-universal-dock-driver<p>Every developer loves to share things about it’s setup. They also loves to make
it better and to spend time on it.</p>
<p>Lorenzo <a href="https://twitter.com/fntlnz">(fntlnz)</a> is super on it! I am
not, plus I bought a Zenbook 3. Super slim, less than 1kg, I can use it to cut
ham probably but the unique USB-C is driving me crazy.</p>
<p>Probably more than the actual 40 degrees that I have in my home office now!
It is probably why I am writing this post btw.</p>
<p>When I bought this laptop 7 months ago the Universal Docker Station was not
available and I wasn’t even able to install linux on this laptop.</p>
<p>Now I have an <a href="https://www.asus.com/Laptops-Accessory/Universal-Dock/">Asus Universal Dock
station</a>. I am feeling a
little bit better but to work it replace a normal charger, it means that without
a socket near you, I can not use a USB… Amazing experience.</p>
<p>I tried other adapter but I didn’t find one good enough. Every one of them had
some input or output port unusable for some reason. Most of them because the
BIOS has a different watt limit and they can not charge the laptop. I never
received a response from ASUS about it. That’s great.</p>
<p>Anyway I am writing this article just as note for myself about the driver that
Lorenzo discover to have the Asus Universal Dock Station’s ethernet port
running.</p>
<p><a href="https://www.realtek.com/DOWNLOADS/downloadsView.aspx?Langid=1&PNid=13&PFid=5&Level=5&Conn=4&DownTypeID=3&GetDown=false">Realtek ethernet
driver</a>.
It’s super easy to install. Just compile it and it will work.</p>
CNCF Italy, first event about opentracingCNCF is a branch of The Linux Foundation focused on Cloud Computing and modern scalable architectures. it's supporting tools like Kubernetes, Prometheus, containerd and so on. If you are using one of them or you are looking to know more about them, this is your meetup. Join us! hashtag CNCFItaly on twitter.https://gianarb.it/img/cncf.jpeg2017-06-05T10:08:27+00:002017-06-05T10:08:27+00:00https://gianarb.it/blog/cncf-italy-first-event<p>CNCF is a branch of The Linux Foundation focused on Cloud Computing and modern
scalable architectures. it’s supporting tools like Kubernetes, Prometheus,
containerd and so on. If you are using one of them or you are looking to know
more about them, this is your meetup. Join us! hashtag #CNCFItaly on twitter.</p>
<p>The event will be 13th July 19.00pm at Toolboox Office in Turin. Reserve your
seat on <a href="https://www.meetup.com/CNCF-Italy/events/241118593/">Meetup.com</a>.</p>
<iframe src="https://www.google.com/maps/embed?pb=!1m18!1m12!1m3!1d2818.7526155267037!2d7.667091951510242!3d45.05024176888683!2m3!1f0!2f0!3f0!3m2!1i1024!2i768!4f13.1!3m3!1m2!1s0x47886d37dd5ababd%3A0x2adc0b0e358ddb6c!2sToolbox+Coworking!5e0!3m2!1sit!2sit!4v1499676857774" width="600" height="450" frameborder="0" style="border:0" allowfullscreen=""></iframe>
<p>This will be a full evening about OpenTracing. OpenTracing <a href="https://www.cncf.io/blog/2016/10/20/opentracing-turning-the-lights-on-for-microservices/">turns the light on
for
microservices</a>.
It is a specification to store and manage trace. How can we follow what’s going
on from the beginning to the end of our requests? What’s happening when they
cross different services? Where is the bottleneck? Tracing helps you to
understand what’s going on. It’s not just for microservices but also for
caching, queue system an so on. Have a <a href="https://trends.google.it/trends/explore?q=opentracing">look at the
trends</a> we need to know
more about it!</p>
<p>Beers, Pizza are offered by CNCF after the two sessions!</p>
<p>Other links:</p>
<ul>
<li><a href="https://www.cncf.io/">CNCF.io</a></li>
<li><a href="https://opentracing.io/">Opentracing</a></li>
<li><a href="https://github.com/openzipkin">OpenZipkin by twitter</a></li>
<li><a href="https://www.youtube.com/watch?v=n8mUiLIXkto">Keynote: OpenTracing and Containers: Depth, Breadth, and the Future of
Tracing - Ben Sigelman</a></li>
</ul>
<h2 id="all-done">All done!</h2>
<p>Amazing event! here some pictures and the video is coming soon!</p>
<div class="slide w3-display-container">
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-sponsor-1.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-1.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-5.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-8.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-9.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-10.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-12.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-13.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-14.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-15.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-16.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-17.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-20.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-21.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-22.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-23.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-24.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-25.jpg" />
<img class="mySlides img-fluid" src="/img/cncf-first/Conf-27.jpg" />
<button class="w3-button w3-display-left" onclick="plusDivs(-1)">❮</button>
<button class="w3-button w3-display-right" onclick="plusDivs(+1)">❯</button>
</div>
<style>
.w3-display-left{position:absolute;top:50%;left:0%;transform:translate(0%,-50%);-ms-transform:translate(-0%,-50%)}
.w3-display-right{position:absolute;top:50%;right:0%;transform:translate(0%,-50%);-ms-transform:translate(0%,-50%)}
.w3-tooltip,.w3-display-container{position:relative}.w3-tooltip .w3-text{display:none}.w3-tooltip:hover .w3-text{display:inline-block}
.w3-btn,.w3-button{border:none;display:inline-block;outline:0;padding:8px 16px;vertical-align:middle;overflow:hidden;text-decoration:none;color:inherit;background-color:inherit;text-align:center;cursor:pointer;white-space:nowrap}
.w3-btn:hover{box-shadow:0 8px 16px 0 rgba(0,0,0,0.2),0 6px 20px 0 rgba(0,0,0,0.19)}
.w3-btn,.w3-button{-webkit-touch-callout:none;-webkit-user-select:none;-khtml-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
.w3-disabled,.w3-btn:disabled,.w3-button:disabled{cursor:not-allowed;opacity:0.3}.w3-disabled *,:disabled *{pointer-events:none}
.w3-btn.w3-disabled:hover,.w3-btn:disabled:hover{box-shadow:none}
.w3-badge,.w3-tag{background-color:#000;color:#fff;display:inline-block;padding-left:8px;padding-right:8px;text-align:center}.w3-badge{border-radius:50%}
.w3-ul{list-style-type:none;padding:0;margin:0}.w3-ul li{padding:8px 16px;border-bottom:1px solid #ddd}.w3-ul li:last-child{border-bottom:none}
.w3-tooltip,.w3-display-container{position:relative}.w3-tooltip .w3-text{display:none}.w3-tooltip:hover .w3-text{display:inline-block}
.w3-ripple:active{opacity:0.5}.w3-ripple{transition:opacity 0s}
.w3-input{padding:8px;display:block;border:none;border-bottom:1px solid #ccc;width:100%}
.w3-select{padding:9px 0;width:100%;border:none;border-bottom:1px solid #ccc}
.w3-dropdown-click,.w3-dropdown-hover{position:relative;display:inline-block;cursor:pointer}
.w3-dropdown-hover:hover .w3-dropdown-content{display:block;z-index:1}
.w3-dropdown-hover:first-child,.w3-dropdown-click:hover{background-color:#ccc;color:#000}
.w3-dropdown-hover:hover > .w3-button:first-child,.w3-dropdown-click:hover > .w3-button:first-child{background-color:#ccc;color:#000}
.w3-dropdown-content{cursor:auto;color:#000;background-color:#fff;display:none;position:absolute;min-width:160px;margin:0;padding:0}
.w3-check,.w3-radio{width:24px;height:24px;position:relative;top:6px}
.w3-sidebar{height:100%;width:200px;background-color:#fff;position:fixed!important;z-index:1;overflow:auto}
.w3-bar-block .w3-dropdown-hover,.w3-bar-block .w3-dropdown-click{width:100%}
.w3-bar-block .w3-dropdown-hover .w3-dropdown-content,.w3-bar-block .w3-dropdown-click .w3-dropdown-content{min-width:100%}
.w3-bar-block .w3-dropdown-hover .w3-button,.w3-bar-block .w3-dropdown-click .w3-button{width:100%;text-align:left;padding:8px 16px}
</style>
<script>
var slideIndex = 1;
showDivs(slideIndex);
function plusDivs(n) {
showDivs(slideIndex += n);
}
function showDivs(n) {
var i;
var x = document.getElementsByClassName("mySlides");
if (n > x.length) {slideIndex = 1}
if (n < 1) {slideIndex = x.length} ;
for (i = 0; i < x.length; i++) {
x[i].style.display = "none";
}
x[slideIndex-1].style.display = "block";
}
</script>
Container security and immutabilityDocker, container and immutability. Have an immutable system has advantages not only from deploy, release and scalability point of view but also from security side. Deploy and build a new release quickly and high frequency improve the way you trust your provisioning system. Have the old environment still running and ready to be rolled back is another good point.https://gianarb.it/img/container-security.png2017-06-05T10:08:27+00:002017-06-05T10:08:27+00:00https://gianarb.it/blog/container-security-immutability<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">55 pages
about how to improve container security. <a href="https://twitter.com/ciliumproject">@ciliumproject</a> <a href="https://twitter.com/hashtag/BPF?src=hash">#BPF</a>, best practices, <a href="https://twitter.com/coreos">@coreos</a> clair, <a href="https://twitter.com/hashtag/apparmor?src=hash">#apparmor</a> <a href="https://t.co/ABiuldYA9b">https://t.co/ABiuldYA9b</a> <a href="https://t.co/61jzWxzb1Y">pic.twitter.com/61jzWxzb1Y</a></p>— :w
!sudo tee % (@GianArb) <a href="https://twitter.com/GianArb/status/871808740080615424">June 5,
2017</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Security is a fascinating topic.
It’s part of every aspect of a system.
From your email server to the HTTP body validation of your API system.</p>
<p>It’s also a very human centric topic. You can use the most stronger security
approaches but if your rules are too hard to follow or too complicated to
implement the end users or your colleagues will become the perfect breach to be
exploited by bad people.</p>
<p>In distributed systems there are interesting challenges like:</p>
<ul>
<li>How can we trust the instances part of the system itself? I mean, how can we
trust a new application after a pool scale?</li>
<li>All the traffic generated by the system needs to be locked. The network
topology grows with the number of services that we add but that’s not a good
excuse to leak on responsibility about how we do manage our network.</li>
</ul>
<p>When you design a system you need to think about security from different points
of view:</p>
<ul>
<li>Security needs to be efficient. This seems obvious but it’s always something
to keep in mind.</li>
<li>It needs to be easy to use in development mode. As we said before. If security
is making things slower somebody will turn it off.</li>
<li>If you are good enough to make it easy, it will be easier to enforce a secure behavior.</li>
</ul>
<p>All this concepts are well applied in all different projects built by the Docker
Community.
Notary, swarmkit are just a few examples but if you think about the update
framework (TUF) and the whole set of things happening behind every <code>docker push</code> and <code>pull</code>
command you suddenly see a great example on how to make complicated things really easy to use.</p>
<p>I published an ebook that you can download for free <a href="/blog/scaledocker">here</a>.
It contains ~55 pages about Container and Docker Security. In this article I
will share one of the concept expressed in that book, <strong>Immutability</strong>.</p>
<p>Docker containers are in fact immutable. This means that a running container
never changes because in case you need to update it, the best practice is to
create a new container with the updated version of your application and delete
the old one.</p>
<p>This aspect is important from different point of views.</p>
<p>Immutability applied to deploy is a big challenge because it opens the door to a very
different set of release strategies like blue green deployment or canary releases.
Immutability also lowers rollback times because you can probably keep the
old version running for a little longer and switch traffic in case of problems.</p>
<p>It’s also a plus from a scalability and stability point of view. For each deploy
you are in fact using provisioning scripts and build tools to package and
release a new version of your application. You are creating new nodes to replace
the old ones that means that you are focused on provisioning and configuration
management. You are justifying all the effort spent to implement infrastructure
as code.</p>
<p>It matters also for security because you will have a fresh container after each
update and in the case of a vulnerability or injection they will be cleaned
during the update.</p>
<p>You have also an instrument to analyse the attacked container with the command
docker diff <container_id> This command shows the differences in the file system.</container_id></p>
<p>It supports 3 events:</p>
<ul>
<li>A - Add</li>
<li>D - Delete</li>
<li>C - Change</li>
</ul>
<p>In case of attack, you can commit the attacked container to analyse it later and
replace it with the original image.</p>
<p>This flow is interesting but if you know that your application does not need to
modify the file system you can use <code>–read-only</code> parameters to make the fs read
only or you can share a volume with the <code>ro</code> suffix <code>-v PWD:/data:ro</code>.</p>
<p>Docker can’t fix the security issues for you, if your application can be
attacked by a code injection then you need to fix your app but Docker offers a
few utilities to make life hard for an hacker and to allow you to have more
control over your environment.</p>
<p>During this chapter we covered some practices and tools that you can follow or
use to build a safe environment.</p>
<p>In general, you need to close your application in an environment that provides
only what you need and what you know.</p>
<p>If your distribution or your container has something that you don’t have under
your control or it is unused then it is a good idea remove these dark points.</p>
<p>That’s all. Immutability is not free and it requires to keep all the tools and
processes involved in deploy, packaging up to speed because all your production
environment depends on these tools. But it’s an important piece of the puzzle.
To read more about tools like Cilium, CoreOS Clair, best practices about
registry and images you can download the pdf <a href="/blog/scaledocker">Play Safe with Docker and
Container Security</a>.</p>
Orbiter an OSS Docker Swarm AutoscalerOrbiter is an open source project design to become a cross provider autoscaler. At the moment it works like Zero Configuration Autoscaler for Docker Swarm. It also has a basic implementation to autoscale Digitalocean. This project is designed with InfluxData a company that provides OSS solution like InfluxDB, Kapacitor and Telegraf. We are going to use all this tools to create an autoscaling policy for your Docker Swarm services.https://gianarb.it/img/swarm.gif2017-04-22T08:08:27+00:002017-04-22T08:08:27+00:00https://gianarb.it/blog/orbiter-docker-swarm-autoscaler<iframe width="560" height="315" src="https://www.youtube.com/embed/Q1xfmfML8ok" frameborder="0" allowfullscreen=""></iframe>
<p>My presentation at the Docker HQ in San Francisco.</p>
<h2 id="autoscaling">Autoscaling</h2>
<p>One of the Cloud’s dreams is a nice world when everything magically happen. You
have unlimited resources and you are just going to use what you need and to pay
to you use.
To do what AWS provides a service called autoscaling-group for example. You can
specify some limitation and some expectation about a group of servers and AWS is
matching your expectation for you.
If you are able to make an automatic provision of a node you can use Cloudwatch
to set some alerts. When AWS trigger these alerts the austocaling-group is
creating or removing one or more instance.</p>
<h3 id="lets-try-with-an-example">let’s try with an example</h3>
<p>You have a web service and you know that for 2 hours every day you don’t need 4
EC2 because you have a lot of traffic, you need 10 of them.
You can create an autoscaling group, set some alerts:</p>
<ol>
<li>When the memory usage is more than 65% for 3 minutes start 3 new servers.</li>
<li>When the memory usage is less than 30% for 5 minutes stop 2 servers.</li>
</ol>
<p>Just to have an idea. In this way AWS knows what do you and you don’t need to
stay in front of your laptop to wait something happen. You can just do something
funny.</p>
<p>It’s something useful, if you think about a daily magazine, they usually has a
lot of traffic in the beginning of the day when all the people are usually
reading news. At that’s an easy scenario.</p>
<p>But it can also happen than a new shared on reddit or HackerNews is getting a
lot of traffic and the last thing that you are looking for is to go down just
during that spike!</p>
<h3 id="actors">Actors</h3>
<p>There are different actors in this comedy. First of all our cluster needs to be
manageable by outside via API. In this example I am going to use Docker Swarm,
Orbiter supports a basic implementation for DigitalOcean but it still requires
some toning.</p>
<p>You need to have some time series database or analytics platform that can
trigger webhook to trigger orbiter based on some metrics.</p>
<p>We ran a demo with the TICKStack (InfluxDB, Telegraf, and Kapacitor) days ago.
It’s available <a href="https://www.influxdata.com/resources/influxdata-helps-docker-auto-scale-monitoring/?ao_campid=70137000000Jgw7">at this
link</a>.</p>
<p>In the end you need to deploy <a href="https://github.com/gianarb/orbiter">orbiter</a>.</p>
<h3 id="orbiter-design-and-arch">Orbiter, design and arch</h3>
<p>Orbiter is an open source tool designed to be a cross platform autoscaler. It is
in go and it provides a REST API to handle scale requests.</p>
<p>It provides one entrypoint:</p>
<pre><code class="language-sh">curl -v -d '{"direction": true}' \
http://localhost:8000/handle/infra_scale/docker
</code></pre>
<ul>
<li><code>direction</code> represent how to scale your service, true means up, false means
down.</li>
<li><code>/handle/infra_scale/docker</code> identify the autoscaling group.
<code>infra_scale</code> is the autoscaler name, <code>docker</code> is the policy name.</li>
</ul>
<p><code>infra_scale</code> for example contains information about the cluster manager, where
it is, what is it? Docker or Digitalocean or what ever?</p>
<p>The policy describes how an application scale. If you know a bit Docker Swarm
<code>docker</code> is the name of the service.</p>
<p>Orbiter supports two different boot methods. One is via configuration:</p>
<pre><code class="language-yaml">autoscalers:
infra_scale:
provider: swarm
parameters:
policies:
docker:
up: 4
down: 3
</code></pre>
<p>The second one is actually only supported by Docker Swarm and it’s called
autodetection. In practice when you start orbiter, it’s looking for a Docker
Swarm up and running. If it finds Swarm it’s going to list all the services
deployed and it’s going to manage all the services labeled with <code>orbiter=true</code>.</p>
<p>By default up and down are set to 1 but you can override them with the label
orbiter.up=3 and orbiter.down=2.</p>
<p>Let’s suppose to have a Docker Swarm cluster with 3 nodes.</p>
<pre><code class="language-bash">$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
11btq767ecqhelidu8ah1osfp * node1 Ready Active Leader
ptre8d4bjccqi6ml6z445u0mz node2 Ready Active
q5rwi3cej9gc1vqyscwfau640 node3 Ready Active
</code></pre>
<p>I deployed a service called <a href="https://github.com/gianarb/micro">gianarb/micro</a>.
It is an open source demo application. There are different versions, I deployed
the version 1.0.0. It only shows the current IP of the container/server.</p>
<pre><code class="language-bash">docker service create --label orbiter=true \
--name micro --replicas 3 \
-p 8080:8000 gianarb/micro:1.0.0
</code></pre>
<p>You can check the number of tasks running with the command:</p>
<pre><code class="language-bash">$ docker service ps micro
ID NAME IMAGE NODE
DESIRED STATE CURRENT STATE ERROR
PORTS
onsqgriv3nel micro.1 gianarb/micro:1.0.0 node3
Running Running 51 seconds ago
yxtxyder7bs3 micro.2 gianarb/micro:1.0.0 node1
Running Running 51 seconds ago
lyzxxdc00052 micro.3 gianarb/micro:1.0.0 node2
Running Running 52 seconds ago
</code></pre>
<p>At this point you can visit port <code>8080</code> of your cluster to have a look of the
service but for this demo doesn’t really matter. We are going to start orbiter
and we are going to trigger a scaling policy to simulate a request made by our
monitoring tool.</p>
<pre><code class="language-bash">docker service create --name orbiter \
--mount type=bind,source=/var/run/docker.sock,destination=/var/run/docker.sock \
-p 8000:8000 --constraint node.role==manager \
-e DOCKER_HOST=unix:///var/run/docker.sock \
gianarb/orbiter daemon --debug
</code></pre>
<p>I am using Docker to deploy orbiter as service. I am using the Unix Socket to
communicate with Docker Swarm and I am deploying this service into the <code>manager</code>
because it needs to have write permission to start and stop tasks. This can be
done only into the manager. You can configure orbiter with the variable
<code>DOCKER_HOST</code> to use REST API. In this way you don’t have this constraint. This
configuration in very easy to show in a demo like this one.</p>
<pre><code class="language-bash">$ docker service logs orbiter
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:24:56Z" level=info
msg="orbiter started"
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:24:56Z" level=debug
msg="Daemon started in debug mode"
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:24:56Z" level=info
msg="Starting in auto-detection mode."
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:24:56Z" level=info
msg="Successfully connected to a Docker daemon"
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:24:56Z" level=debug
msg="autodetect_swarm/micro added to orbiter. UP 1, DOWN 1"
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:24:56Z" level=info
msg="API Server run on port :8000"
</code></pre>
<p>As you can see into the logs the API are running on port 8000 and orbiter
already detected a service called <code>micro</code>, the one that we deployed before and
it auto-created a autoscaling group called <code>autodetection_swarm/micro</code>.
This is the unique name that we can use when we trigger our scale request.</p>
<pre><code class="language-bash">$ curl -d '{"direction": true}' -v
http://10.0.57.3:8000/handle/autodetect_swarm/micro
* Trying 10.0.57.3...
* TCP_NODELAY set
* Connected to 10.0.57.3 (10.0.57.3) port 8000 (#0)
> POST /handle/autodetect_swarm/micro HTTP/1.1
> Host: 10.0.57.3:8000
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Length: 19
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 19 out of 19 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Tue, 18 Apr 2017 09:30:35 GMT
< Content-Length: 0
<
* Curl_http_done: called premature == 0
* Connection #0 to host 10.0.57.3 left intact
</code></pre>
<p>With that cURL I simulated a scale request and as you can see in the log above
orbiter detected the request and it scaled up 1 task for our service called
<code>macro</code></p>
<pre><code class="language-bash">$ docker service logs orbiter
orbiter.1.zop1qkwa1qxy@node1 | POST /handle/autodetect_swarm/micro HTTP/1.1
orbiter.1.zop1qkwa1qxy@node1 | Host: 10.0.57.3:8000
orbiter.1.zop1qkwa1qxy@node1 | Accept: */*
orbiter.1.zop1qkwa1qxy@node1 | Content-Length: 19
orbiter.1.zop1qkwa1qxy@node1 | Content-Type:
application/x-www-form-urlencoded
orbiter.1.zop1qkwa1qxy@node1 | User-Agent: curl/7.52.1
orbiter.1.zop1qkwa1qxy@node1 |
orbiter.1.zop1qkwa1qxy@node1 | {"direction": true}
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:30:35Z" level=info
msg="Received a new request to scale up micro with 1 task." direc
tion=true service=micro
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:30:35Z" level=debug
msg="Service micro scaled from 3 to 4" provider=swarm
orbiter.1.zop1qkwa1qxy@node1 | time="2017-04-18T09:30:35Z" level=info
msg="Service micro scaled up." direction=true service=micro
</code></pre>
<p>We can verify the current number of tasks that are running for <code>micro</code> and we
can see that it’s not 3 as before but 4.</p>
<pre><code class="language-bash">$ docker service ls
ID NAME MODE REPLICAS
IMAGE
azi8zyeor5eb micro replicated 4/4
gianarb/micro:1.0.0
ezklgb6uak8b orbiter replicated 1/1
gianarb/orbiter:latest
</code></pre>
<p>This project is open source on
<a href="https://github.com/gianarb/orbiter">github.com/gianarb/orbiter</a> you can have a look on
it, try and leave some feedback or request if you need something different.</p>
<p>PR are also open if you are working with a different cluster manager or with a
different provider, add a new one is very easy. It’s just a new interface to
implement.</p>
LinuxKit operating system built for containerLinuxKit is a new tool presented during the DockerCon 2017 built by Docker to manage cross architecture and cross kernel testing. LinuxKit is a secure, portable and lean operating system built for containers. It supports different hypervisor as MacOS hyper or QEMU to run testsuite on different architectures. In this article I am showing you some basic concept above this tool. How it works and why it can be useful.https://gianarb.it/img/builder.gif2017-04-18T10:08:27+00:002017-04-18T10:08:27+00:00https://gianarb.it/blog/linuxkit-operating-system-build-for-containers<p>Linuxkit is a new project presented by Docker during the DockerCon 2017. If we
look at the description of the project on
<a href="https://github.com/linuxkit/linuxkit">GitHub</a>:</p>
<blockquote>
<p>A secure, portable and lean operating system built for containers</p>
</blockquote>
<p>I am feeling already exited. I was an observer of the project when <a href="https://twitter.com/justincormack">Justin
Cormack</a> and the other
<a href="https://github.com/linuxkit/linuxkit/graphs/contributors">contributors</a> was
working on a private repository. I was invited as part of ci-wg group into the
CNCF and I loved this project from the first day.</p>
<p>You can think about linuxkit as a builder for Linux operating system everything
based on containers.</p>
<p>It’s a project that can stay behind your continuous integration system to allow
us to test on different kernel version and distribution. You can a light kernels
with all the services that you need and you can create different outputs
runnable on cloud providers as Google Cloud Platform, with Docker or with QEMU.</p>
<h2 id="continuous-delivery-new-model">Continuous delivery, new model</h2>
<p>I am not really confident about Google Cloud Platform but just to move over I am
going to do some math with AWS as provider.
Let’s suppose that I have the most common continuous integration system, one big
box always up an running configured to support all your projects or if you are
already good you are running containers to have separated and isolated
environment.</p>
<p>Let’s suppose that you Jenkins is running all times on m3.xlarge:</p>
<p><code>m3.xlarge</code> used 100% every months costs 194.72$.</p>
<p>Let’s have a dream. You have a very small server with just a frontend
application for your CI and all jobs are running in a separate instance, tiny as
a t2.small.</p>
<p><code>t2.small</code> used only 1 hour costs 0.72$ .</p>
<p>I calculated 1 hour because it’s the minimum that you can pay and I hope that
your CI job can run for less than 1 hour.
Easy math to calculate the number of builds that you need to run to pay as you
was paying before.</p>
<p>194.72 / 0.72 ~ 270 builds every month.</p>
<p>If you are running less than 270 builds a months you can save some money
too. But you have other benefits:</p>
<ol>
<li>More jobs, more instances. Very easy to scale. Easier that Jenkins
master/slave and so on.</li>
<li>How many times during holidays your Jenkins is still up and running without
to have nothing to do? During these days you are just paying for the frontend
app.</li>
</ol>
<p>And these are just the benefit to have a different setup for your continuous
delivery.</p>
<h2 id="linuxkit-ci-implementation">LinuxKit CI implementation</h2>
<p>There is a directory called
<a href="https://github.com/linuxkit/linuxkit/tree/master/test">./test</a> that contains
some linuxkit use case but I am going to explain in practice how linuxkit is
tested. Because it uses itself, awesome!</p>
<p>In first you need to download and compile linuxkit:</p>
<pre><code class="language-shell">git clone github.com:linuxkit/linuxkit $GOPATH/src/github.com/linuxkit/linuxkit
make
./bin/moby
</code></pre>
<p>You can move it in your <code>$PATH</code> with <code>make install</code>.</p>
<pre><code>$ moby
Please specify a command.
USAGE: moby [options] COMMAND
Commands:
build Build a Moby image from a YAML file
run Run a Moby image on a local hypervisor or remote cloud
version Print version information
help Print this message
Run 'moby COMMAND --help' for more information on the command
Options:
-q Quiet execution
-v Verbose execution
</code></pre>
<p>At the moment the CLI is very simple, the most important commands are build and
run. linuxkit is based on YAML file that you can use to describe your kernel,
with all applications and all the services that you need. Let’s start with the
<a href="https://github.com/linuxkit/linuxkit/blob/master/test/test.yml">linuxkit/test/test.yml</a>.</p>
<pre><code class="language-yaml">kernel:
image: "mobylinux/kernel:4.9.x"
cmdline: "console=ttyS0"
init:
- mobylinux/init:8375addb923b8b88b2209740309c92aa5f2a4f9d
- mobylinux/runc:b0fb122e10dbb7e4e45115177a61a3f8d68c19a9
- mobylinux/containerd:18eaf72f3f4f9a9f29ca1951f66df701f873060b
- mobylinux/ca-certificates:eabc5a6e59f05aa91529d80e9a595b85b046f935
onboot:
- name: dhcpcd
image: "mobylinux/dhcpcd:0d4012269cb142972fed8542fbdc3ff5a7b695cd"
binds:
- /var:/var
- /tmp:/etc
capabilities:
- CAP_NET_ADMIN
- CAP_NET_BIND_SERVICE
- CAP_NET_RAW
net: host
command: ["/sbin/dhcpcd", "--nobackground", "-f", "/dhcpcd.conf", "-1"]
- name: check
image: "mobylinux/check:c9e41ab96b3ea6a3ced97634751e20d12a5bf52f"
pid: host
capabilities:
- CAP_SYS_BOOT
readonly: true
outputs:
- format: kernel+initrd
- format: iso-bios
- format: iso-efi
- format: gcp-img
</code></pre>
<p>Linuxkit builds everythings inside a container, it means that you don’t need a
lot of dependencies it’s very easy to use. It generates different <code>output</code> in
this case <code>kernel+initrd</code>, <code>iso-bios</code>, <code>iso-efi</code>, <code>gpc-img</code> depends of the
platform that you are interested to use to run your kernel.</p>
<p>I am trying to explain a bit how this YAML works. You can see that there are
different primary section: <code>kernel</code>, <code>init</code>, <code>onboot</code>, <code>service</code> and so on.</p>
<p>Pretty much all of them contains the keyword <code>image</code> because as I said before
everything is applied on containers, in this example they are store in
<a href="https://hub.docker.com/u/mobylinux/">hub.docker.com/u/mobylinux/</a>.</p>
<p>The based kernel is <code>mobylinux/kernel:4.9.x</code>, I am just reporting what the
<a href="https://github.com/linuxkit/linuxkit#yaml-specification">README.md</a> said:</p>
<ul>
<li><code>kernel</code> specifies a kernel Docker image, containing a kernel and a
filesystem tarball, eg containing modules. The example kernels are built from
<code>kernel/</code></li>
<li><code>init</code> is the base <code>init</code> process Docker image, which is unpacked as the base
system, containing <code>init</code>, <code>containerd</code>, <code>runc</code> and a few tools. Built from
<code>pkg/init/</code></li>
<li><code>onboot</code> are the system containers, executed sequentially in order. They
should terminate quickly when done.</li>
<li><code>services</code> is the system services, which normally run for the whole time the
system is up</li>
<li><code>files</code> are additional files to add to the image</li>
<li><code>outputs</code> are descriptions of what to build, such as ISOs.</li>
</ul>
<p>At this point we can try it. If you are on MacOS as I was you don’t need to
install anything one of the runner supported by <code>linuxkit</code> is <code>hyperkit</code> it
means that everything is available in your system.</p>
<p><code>./test</code> contains different test suite but now we will stay focused on
<code>./test/check</code> directory. It contains a set of checks to validate how the
kernel went build by LinuxKit. They are the smoke tests that are running on each
new pull request created on the repository for example.</p>
<p>As I said everything runs inside a container, if you look into the check
directory there is a makefile that build a mobylinux/check image, that image
went run in LinuxKit, into the <code>test.yml</code> file:</p>
<pre><code class="language-yaml">onboot:
- name: check
image: "mobylinux/check:c9e41ab96b3ea6a3ced97634751e20d12a5bf52f"
pid: host
capabilities:
- CAP_SYS_BOOT
readonly: true
</code></pre>
<p>You can use the
<a href="https://github.com/linuxkit/linuxkit/blob/master/test/check/Makefile">Makefile</a>
inside the check directory to build a new version of check, you can just use
the command <code>make</code>.</p>
<p>When you have the right version of your test we can build the image used by moby:</p>
<pre><code>cd $GOPATH/src/github.com/linuxkit/linuxkit
moby build test/test.yml
</code></pre>
<p>Part of the output is:</p>
<pre><code class="language-shell">Create outputs:
test-bzImage test-initrd.img test-cmdline
test.iso
test-efi.iso
test.img.tar.gz
</code></pre>
<p>And if you look into the directory you can see that there are all these files
into the root. These files can be run from qemu, google cloud platform,
hyperkit and so on.</p>
<pre><code class="language-shell">moby run test
</code></pre>
<p>On MacOS with this command LinuxKit is using hyperkit to start a VM, I can not copy
paste all the output but you can see the hypervisor logs:</p>
<pre><code>virtio-net-vpnkit: initialising, opts="path=/Users/gianlucaarbezzano/Library/Containers/com.docker.docker/Data/s50"
virtio-net-vpnkit: magic=VMN3T version=1 commit=0123456789012345678901234567890123456789
Connection established with MAC=02:50:00:00:00:04 and MTU 1500
early console in extract_kernel
input_data: 0x0000000001f2c3b4
input_len: 0x000000000067b1e5
output: 0x0000000001000000
output_len: 0x0000000001595280
kernel_total_size: 0x000000000118a000
booted via startup_32()
Physical KASLR using RDRAND RDTSC...
Virtual KASLR using RDRAND RDTSC...
Decompressing Linux... Parsing ELF... Performing relocations... done.
Booting the kernel.
[ 0.000000] Linux version 4.9.21-moby (root@84baa8e89c00) (gcc version 6.2.1 20160822 (Alpine 6.2.1) ) #1 SMP Sun Apr 9 22:21:32 UTC 2017
[ 0.000000] Command line: earlyprintk=serial console=ttyS0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] x86/fpu: Using 'eager' FPU context switches.
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000003fffffff] usable
</code></pre>
<p>When the VM is ready LinuxKit is starting all the <code>init</code>, <code>onboot</code>, the logs is
easy to understand as the <code>test.yml</code> is starting <code>containerd</code>, <code>runc</code>:</p>
<pre><code>init:
- mobylinux/init:8375addb923b8b88b2209740309c92aa5f2a4f9d
- mobylinux/runc:b0fb122e10dbb7e4e45115177a61a3f8d68c19a9
- mobylinux/containerd:18eaf72f3f4f9a9f29ca1951f66df701f873060b
- mobylinux/ca-certificates:eabc5a6e59f05aa91529d80e9a595b85b046f935
onboot:
- name: dhcpcd
image: "mobylinux/dhcpcd:0d4012269cb142972fed8542fbdc3ff5a7b695cd"
binds:
- /var:/var
- /tmp:/etc
capabilities:
- CAP_NET_ADMIN
- CAP_NET_BIND_SERVICE
- CAP_NET_RAW
net: host
command: ["/sbin/dhcpcd", "--nobackground", "-f", "/dhcpcd.conf", "-1"]
- name: check
image: "mobylinux/check:c9e41ab96b3ea6a3ced97634751e20d12a5bf52f"
pid: host
capabilities:
- CAP_SYS_BOOT
readonly: true
</code></pre>
<pre><code>Welcome to LinuxKit
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\_______/
/ # INFO[0000] starting containerd boot... module=containerd
INFO[0000] starting debug API... debug="/run/containerd/debug.sock" module=containerd
INFO[0000] loading monitor plugin "cgroups"... module=containerd
INFO[0000] loading runtime plugin "linux"... module=containerd
INFO[0000] loading snapshot plugin "snapshot-overlay"... module=containerd
INFO[0000] loading grpc service plugin "healthcheck-grpc"... module=containerd
INFO[0000] loading grpc service plugin "images-grpc"... module=containerd
INFO[0000] loading grpc service plugin "metrics-grpc"... module=containerd
</code></pre>
<p>The last step is the <code>check</code> that runs the real test suite:</p>
<pre><code>kernel config test succeeded!
info: reading kernel config from /proc/config.gz ...
Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
........
.......
Moby test suite PASSED
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\_______/
[ 3.578681] ACPI: Preparing to enter system sleep state S5
[ 3.579063] reboot: Power down
</code></pre>
<p>The last log is the output of
<a href="https://github.com/linuxkit/linuxkit/blob/master/test/check/check-kernel-config.sh">check-kernel-config.sh</a>
files.</p>
<p>If you are on linux you can do the same command but by the default you are going
to use <a href="https://www.qemu-project.org/">qemu</a> an open source machine emulator.</p>
<pre><code class="language-bash">sudo apt-get install qemu
</code></pre>
<p>I did some test in my Asus Zenbook with Ubuntu, when you run <code>moby run</code> this is
the command executed with qemu:</p>
<pre><code>/usr/bin/qemu-system-x86_64 -device virtio-rng-pci -smp 1 -m 1024 -enable-kvm
-machine q35,accel=kvm:tcg -kernel test-bzImage -initrd test-initrd.img -append
console=ttyS0 -nographic
</code></pre>
<p>By default is testing on <code>x86_64</code> but qemu supports a lot of other archs and
devices. You can simulate an arm and a rasperry pi for example. At the
moment LinuxKit is not ready to emulate other architecture. But this is the main
scope for this project. It’s just a problem of time. It will be able soon!</p>
<p>Detect if the build succeed or failed is not easy as you probably expect. The
status inside the VM is not the one that you get in your laptop. At the moment
to understand if the code in your PR is good or bad we are parsing the output:</p>
<pre><code>define check_test_log
@cat $1 |grep -q 'Moby test suite PASSED'
endef
</code></pre>
<p><a href="https://github.com/linuxkit/linuxkit/blob/master/Makefile">./linuxkit/Makefile</a></p>
<p>Explain how linuxkit tests itself at the moment is the best way to get how it
works. It is just one piece of the puzzle, if you have a look here <a href="https://github.com/linuxkit/linuxkit/pulls">every
pr</a> has a GitHub Status that point to
a website that contains logs related that particular build. That part is not
managed by linuxkit because it’s only the builder used to create the
environment. All the rest is managed by
<a href="https://github.com/docker/datakit">datakit</a>. I will speak about it probably in
another blogpost.</p>
<h2 id="conclusion">Conclusion</h2>
<p>runc, docker, containerd, rkt but also Prometheus, InfluxDB, Telegraf a lot of
projects supports different architecture and they need to run on different
kernels with different configuration and capabilities. They need to run on your
laptop, in your IBM server and in a Raspberry Pi.</p>
<p>This project is in an early state but I understand why Docker needs something
similar and also, other projects as I said are probably going to get some
benefits from a solution like this one. Have it open source it’s very good and
I am honored to be part of the amazing group that put this together. I just did
some final tests and I tried to understand how it’s designed and how it works.
This is the result of my test. I hope that can be helpful to start in the right
mindset.</p>
<p>My plan is to create a configuration to test InfluxDB and play a bit with <code>qemu</code>
to test it on different architectures and devices. Stay around a blogpost will
come!</p>
<p>Some Links:</p>
<ul>
<li><a href="https://blog.docker.com/2017/04/introducing-the-moby-project/">INTRODUCING MOBY PROJECT: A NEW OPEN-SOURCE PROJECT TO ADVANCE THE SOFTWARE
CONTAINERIZATION MOVEMENT</a></li>
<li><a href="https://github.com/linuxkit">github.com/linuxkit</a></li>
<li><a href="https://github.com/moby">github.com/moby</a></li>
</ul>
<p class="text-muted">
Reviewers: <a href="https://twitter.com/justincormack">Justin Cormack</a>
</p>
<div class="post row">
<div class="col-md-12">
<div class="bs-callout bs-callout-info row">
<div class="row">
<div class="col-md-12">
<h2><a href="//gianarb.it/blog/docker-the-fundamentals" target="_blank">get "Docker the Fundamentals"</a> <small>by. Drive your boat as a Captain</small></h2>
</div>
</div>
<div class="row">
<div class="col-md-3">
<a href="//gianarb.it/blog/docker-the-fundamentals" target="_blank"><img src="/img/the-fundamentals.jpg" class="img-fluid" /></a>
</div>
<div class="col-md-9">
<p>
You can get the Chapter 2 of the book <a href="/blog/scaledocker" target="_blank">"Drive your boat as a Captain"</a> just leave click on the
cover and leave your email to receive a free copy.</p>
<p>This chapter is getting started with Docker Engine and the basic
concept around registry, pull, push and so on. It's a good way to start from
zero with Docker.</p>
</div>
</div>
</div>
</div>
</div>
Containers why we are hereCloud computing, containers, devops, everything is moving so fast that sometime for big companies or CTO is very hard keep track of everything. What it's just a new trend and what I really need. This post contains my options and a bit of history about docker, cloud computing, aws and containers.https://gianarb.it/img/container-security.png2017-03-12T08:08:27+00:002017-03-12T08:08:27+00:00https://gianarb.it/blog/containers-why-we-are-here<blockquote>
<p>“It is change, continuing change, inevitable change, that is the dominant
factor in society today. No sensible decision can be made any longer without
taking into account not only the world as it is, but the world as it will be…
This, in turn, means that our statesmen, our businessmen, our everyman must take
on a science fictional way of thinking” Asimov, 1981</p>
</blockquote>
<h1 id="isolation-and-virtualization">Isolation and Virtualization</h1>
<p>I can see clearly two kind of invention: the ones that allow people to do
something they couldn’t do before and the ones that let them do something
better. Fire, for example, gave people the chance to cook food, push away wild
beasts and warm themselves up during cold nights. Many years later, electricity
let people warm their houses just by pushing a button. After wheels discovery
people began to travel and to trades goods, but was only with car’s invention
that they might do it faster and efficiently. Similarly, the web creates a huge
network, able to connect people all over the world, web application gave people
tools to use and customise such a complex system. Under this perspective,
container is one of the main revolution of the last years, a unique tool that
helps with app management and development. Let’s discover something more about
the real story of containers.</p>
<p>We have not a lot of documentation about why Bill Joy 18th March 1982 added
chroot into the BSD probably to emulate him solutions and program is an isolated
root. That’s was amazing but not enough few years later in 1991 Bill Cheswick
extended chroot with security features provided by FreeBSD and implemented the
“jails” and in 2000 he introduced what we know as the proper jails command now
our chroots can not be anything, anywhere out of themself. When you start a
process in a chroot the PID is one and there is only that process but from
outside you can see all processes that are running in a chroot. Our
applications can not stay in a jail! They need to communicate with outside,
exchange information and so on. To solve this problem in 2002 in the kernel
version 2.4.19 a group of developers like Eric W. Biederman, Pavel Emelyanov
introduced the namespace feature to manage system resources like network,
process and file system.</p>
<p>This is just a bit of history about how the ecosystem spin up, in the end of
this chapter we will try to understand how why Docker arrives on the scene, but
the main goal of this book is on another layer and on another complexity, we are
here to understand how manage all this things in cloud and how to design a
distributed system but you know the past is important to build a solid future.</p>
<p>All this great features are now popular under the name of container, nothing
really news and this is one of the reason about why all this things are amazing!
They are under the hood from a while! Solid and tested feature put together and
made usable.</p>
<p>Nothing to say about the importance for a system to being isolated: isolation
helps us to usefully manage resources, security and monitoring, in the best way,
false problems creation in specific applications, often not even related to our
app.</p>
<p>The most common solution is virtualization: you can use an hypervisor to create
virtual server in a single machine. There are different kind of virtualization:</p>
<ul>
<li>Full virtualization</li>
<li>Para virtualization like Virtual Machine, Xen, VMware</li>
<li>Operating System virtualization like Containers</li>
<li>Application virtualization like JVM.</li>
</ul>
<p><img class="img-fluid" src="/img/virtualization.png" />
<a href="https://fntlnz.wtf/post/why-containers/" target="_blank"><small>img from fntlnz’s blog. Thanks</small></a></p>
<p>The main differences between them is how they abstract the layers, application,
processing, network, storage and also about how the superior level interact with
underlying level. For example into the Full virtualization the hardware is
virtualized, into the para virtualization not.</p>
<p>Container is an operation-system-level virtualization. The main difference
between Container and Virtual Machine is the layer: the first works on the
operating system, the second on the hardware layer.</p>
<p>When we speak about container we are focused on the application virtualization
and on a specific feature provided by the kernel called Linux Containers (LXC):
what we do when we build containers is create new isolated Linux systems into
the same host, it means that we can not change the operation system for example
because our virtualization layer doesn’t allow us to run Linux containers out of
Linux.</p>
<h1 id="the-reasons">The reasons</h1>
<p>Revolutions are not related to a single and specific event but come from
multiple movements and changes: Container is just a piece of the story.</p>
<p>Cloud Computing allowed us to think about our infrastructure as an instable
number of servers that can scale up and down, in a reasonable short amount of
time, with less money and without the investment requested to manage a big
infrastructure made of more than one datacenter across the world.</p>
<p>As a consequence, applications that had been in a cellar, now are on Amazon Web
Service, with a load balancer and maybe different availability zone. This
allowed little teams and medium companies, without datacenter and
infrastructures, to think about concept like distribution, high availability,
redundancy. Evolution never stop .</p>
<p>Once our applications are running in few virtual machines, our business will
grow up so we start to scale up and down this servers to serve all our users.
We experimented few benefits but also a lot of issues related, for example, to
the time requested for managing this dynamism; moreover big applications are
usually more expensive to scale.</p>
<p>Our application can only grow but the deploy can be really expensive. We
discovered that the behavior of an application is not the same across all of our
services and entrypoint, because few of them receive more traffic that others.
So, we started to split our big applications in order to make them easy to scale
and monitor. The problem was that, in order to maintain our standard, we need to
find a way to keep them isolated, safe and able to communicate each others.</p>
<p>The Microservices Architecture arrived and companies like Netflix, Amazon,
Google and others counts hundreds and hundreds of little and specific of
services that together work to serve big and profitable products. Netlix is one
of first companies that started sharing the way they build Netlix.com: with more
that 400 microservices, they managed feature like registration, streaming,
rankins and all what the application provides. At the moment, Containers are
the best solution for managing a dense and dynamic environment with a good
control, security and for moving your application between servers.</p>
<p class="text-muted">
Reviewers: Arianna Scarcella, <a href="https://twitter.com/TheBurce">Jenny Burcio</a>
</p>
About your images, security tipsEverything unnecessary in your system could be a very stupid vulnerability. We already spoke about this idea in the capability chapter and the same rule exists when we build an image. Having tiny images with only what our application needs to run is not just a goal in terms of distribution but also in terms of cost of maintenance and security.https://gianarb.it/img/container-security.png2016-12-28T08:08:27+00:002016-12-28T08:08:27+00:00https://gianarb.it/blog/about-your-images-security-tips<p>Everything unnecessary in your system could be a very stupid vulnerability. We
already spoke about this idea in the capability chapter and the same rule
exists when we build an image. Having tiny images with only what our
application needs to run is not just a goal in terms of distribution but also
in terms of cost of maintenance and security. If you have some small
experience with docker already you probably know the
<a href="https://hub.docker.com/_/alpine/">alpine</a> image. It is build
from the Alpine distribution and it’s only 5MB size, if your application can
run inside it then this is a very good optimization that you can do. What
about your binaries? Can your application run standalone? If the answer is yes
you can think about a very very minimal image. scratch is usually used as a
base for other images like debian and ubuntu but you can also use it to run
your golang binary and let me show you something with our micro application.
In the <a href="https://github.com/gianarb/micro/releases/tag/1.0.0">release page</a>,
there are a list of binaries already compiled and ready to be used. In this
case we can download the linux_386 binary.</p>
<p><img class="img-fluid" src="/img/security-image/micro-release.png" /></p>
<pre><code class="language-bash">curl -SsL https://github.com/gianarb/micro/releases/download/1.0.0/micro_1.0.0_linux_386 > micro
</code></pre>
<p>And we know we can include this binary in the scratch image with this Dockerfile</p>
<pre><code class="language-bash">FROM scratch
ADD ./micro /micro
EXPOSE 8000
CMD ["/micro"]
</code></pre>
<pre><code class="language-bash">docker build -t micro-scratch .
docker run -p 8000:8000 micro-scratch
</code></pre>
<p>The expectation is an http application on port 8000 but the main difference is
the size of the image, the old one from alpine is 12M the new one is 5M.</p>
<p>The scratch image is impossibile to use with all applications but if you have a
binary you can remove a lot of unused overhead.</p>
<p>Another way to understand the status of your image is to scan it to detect
security vulnerabilities or exposures. Docker Hub and Docker Cloud can do it
for private images. This is a great feature to have in your pipeline to scan
an image after a build.</p>
<p>CoreOS provides an open source project called <a href="https://github.com/coreos/clair">clair</a> to do the same in your environment.</p>
<p>It is an application in Golang that exposes a set of HTTP API to
pull, push and analyse images. It downloads vulnerabilities from different
sources like <a href="https://security-tracker.debian.org/tracker">Debian Security
Tracker</a> or <a href="https://www.redhat.com/security/data/metrics/">RedHat Security
Data</a>. Each vulnerability is
stored in Postgres. Clair works like static analyzer, this means that it
doesn’t need to run our container to scan it but it persists different checks
directly into the filesystem of the image.</p>
<pre><code class="language-bash">docker run -it -p 5000:5000 registry
</code></pre>
<p>With this command we are running a private registry to use as a source for the
image to scan</p>
<pre><code class="language-bash">docker pull gianarb/micro:1.0.0
docker tag gianarb/micro:1.0.0 localhost:5000/gianarb/micro:1.0.0
docker push localhost:5000/gianarb/micro:1.0.0
</code></pre>
<p>Now that we pushed in our private repo the micro image we can setup clair.</p>
<pre><code class="language-bash">mkdir $HOME/clair-test/clair_config
cd $HOME/clair-test
curl -L https://raw.githubusercontent.com/coreos/clair/v1.2.2/config.example.yaml -o clair_config/config.yaml
curl -L https://raw.githubusercontent.com/coreos/clair/v1.2.2/docker-compose.yml -o docker-compose.yml
</code></pre>
<p>Modify <code>$HOME/clair_config/config.yml</code> and add the proper source
<code>postgresql://postgres:password@postgres:5432?sslmode=disable</code></p>
<p>Now you can run the following command to start postgres and clair:</p>
<pre><code class="language-bash">docker-compose up
</code></pre>
<p>To make our test easier, we will use another CLI called hyperclair that is just
a client to work with this application. If you are using Mac OS, you can follow
the above commands, if you are in another OS you can find the correct url in
the release page</p>
<pre><code class="language-bash">curl -SSl https://github.com/wemanity-belgium/hyperclair/releases/download/0.5.2/hyperclair-darwin-386 > ~/hyperclair
chmod 755 ~/hyperclair
</code></pre>
<p>Now we have an executable in ~/hyperclair</p>
<pre><code class="language-bash">~/hyperclair pull localhost:5000/gianarb/micro:1.0.0
~/hyperclair push localhost:5000/gianarb/micro:1.0.0
~/hyperclair analyze localhost:5000/gianarb/micro:1.0.0
~/hyperclair report localhost:5000/gianarb/micro:1.0.0
</code></pre>
<p>The generated report looks like this:</p>
<p><img class="img-fluid" src="/img/security-image/report-clair.png" /></p>
<p>Hyperclair is just one of the implementations of clair, you can decide to use
it or build your own implementation in your pipeline.</p>
Docker registry to ship and manage your containers.Build and Run containers is important but ship them out of your laptop is the best part! A Registry is used to store and manage your images and all your layers. You can use a storage to upload and download them across your servers and to share them with your colleagues.https://gianarb.it/img/docker.png2016-12-14T08:08:27+00:002016-12-14T08:08:27+00:00https://gianarb.it/blog/docker-registry-to-ship-your-containers<p>Build and Run containers is important but ship them out of your laptop is the
best part! The Registry is a very important tool that requires a bit more
attention. A Registry is used to store and manage your images and all your
layers. You can use a storage to upload and download them across your servers
and to share them with your colleagues.</p>
<p>The most popular one is hub.docker.com
it contains different kind of images: public, official and private. You can
create an account and push your images or build them for example from a github
or bitbucket repository. The integration with GitHub and Bitbucket is called
“Automated Builds”. it allows you to create a continuous integration
environment for your images, when you select “Create” and “Automated Builds”
you can specify a repository and a path of your Dockerfile. You can specify
more that one path from the same repository to build more that one image tag.
In this way you can centralize and build your images every time that a new
change is pushed into the repository. It also supports organizations to split
your images in different groups and manage visibility of them in case of
private images.</p>
<p>By default any developer can push their image to registry and
they’ll be public and free for other developers to use. Official images are
those public images selected and maintained from specific organization or
member of the communities, the idea is that they have a better quality or who
provides them are usually involved into the product development. A set of
official images are: Nginx, Redis, MySql, PHP, Go and so on
<a href="https://hub.docker.com/explore">https://hub.docker.com/explore</a>.</p>
<p>Docker Hub offers different plan to store
private images, all people has one for free but if you need more you can pay a
plan and store more.</p>
<p>Registry is not just a tool but it’s a specification, it
describe how expose capabilities has pull, push, search and so on. This
solution allowed the ecosystem to implement these rules in other projects and
save the compatibility with the Docker Client and with the other runtime engine
that use this capability. It’s for this reason that other providers as
Kubernetes, Cloud Foundry supports download from Docker Hub. This specification
has 2 version, v1 and v2 the most famous registries implement both standard and
they fallback from v2 to v1 for features that are not supported yet. For
example Search is not supported at the moment into the v2 but only in v1.</p>
<p>If you are looking for an In-House solution you have different tools available
online. The first one is distribution. It is provided by Docker, it’s open
source and offers a very tiny registry that you can start and store in your
server. It also supports different storage like the local filesystem and S3.
This feature is very interesting because usually the size of the images and the
number of layers increase very fast and you also need to keep them safe with
backup and redundancy policies for high availability. This is very important if
your environment is based on containers it means that your register is a core
part of your company. Let’s start a Docker Distribution:</p>
<pre><code class="language-bash">$ docker run -d -p 5000:5000 --name registry registry:2
</code></pre>
<p>In docker the default registry is hub.docker.com it means that when we push or
pull an image we are reaching this registry:</p>
<pre><code class="language-bash">$ docker pull alpine
</code></pre>
<p>To push our images in another registry we need to tag them:</p>
<pre><code class="language-bash">$ docker tag alpine 127.0.0.1:5000/alpine
</code></pre>
<p>With this command you tagged the alpine to a registry 127.0.0.1:5000 because as
we said in previous chapters the name of the image contains a lot of
information:</p>
<pre><code>REGISTRY/NAME:VERSION
</code></pre>
<p>The default registry is hub.docker.com a name could be simple as alpine or with
a username matt/alpine and you can pin a specific build with a version you can
use semver or for example the sha of the commit the default VERSION is latest.</p>
<p>Now that we have a new tag we can push and pull it in from our registry:</p>
<pre><code class="language-bash">$ docker push 127.0.0.1:5000/alpine
$ docker pull 127.0.0.1:5000/alpine
</code></pre>
<p>A very important information to remember when you start a customer registry is
that every layers, every build is stored and it’s very easy to have a big
registry, you need to monitor the instance to be sure that your server has
enough disks space and also take care about high availability. In a real
environment the registry it the core of your infrastructure, developers use it
to pull and push build and also to put a version in production. Take care of
your registry.</p>
<p>Other that Docker provided registry there are few alternatives. <a href="https://www.sonatype.com/nexus-repository-sonatype">Nexus</a> is a registry manager that
support a lot of languages and packages if you are a Java developer you know
it. Nexus supports Docker Registry API v1 and v2. The Docker registry
specification is young but it has 2 version already.</p>
<p>We can use the image provided by Sonatype and start our Nexus repository:</p>
<pre><code class="language-bash">$ docker run -d -p 8082:8082 -p 8081:8081 \
-v /tmp/sonata:/sonatype-work --name nexus sonatype/nexus3
$ docker logs -f nexus
</code></pre>
<p>When our log tells us that Nexus is ready we can reach the ui from our browser
http://localhost:8081/ or with the IP of your Docker Machine if you are using
Docker for Mac/Windows or Docker in Linux. The default credentials are username
admin and password admin123.</p>
<p><img class="img-fluid" src="/img/docker-registry/nexus-image-loaded.png" /></p>
<p>First of all we need to create a new Hosted Repository for Docker, we need to
press the Settings Icon top left of the page, Repositories and Create
Repository. I called mine mydocker and you need to specify an HTTP port for
that repository, we shared port 8082 during the run and for this reason I chose
8082.</p>
<p><img class="img-fluid" src="/img/docker-registry/nexus-create-repo.png" /></p>
<p>Nexus has different kind of repositories Host means that it’s self hosted but
you can also create a Proxy Repository to proxy for example the official Docker
Hub.
Now we need to login to out docker registry:</p>
<pre><code class="language-bash">$ docker login 127.0.0.1:8082
</code></pre>
<p>Now we can tag an alpine and push the tag into the repository</p>
<pre><code class="language-bash">$ docker tag alpine 127.0.0.1:8082/alpine
$ docker push 127.0.0.1:8082/alpine
</code></pre>
<p>You can go in Assets click on mydocker repository and see that your image is
correctly stored.</p>
<p><a href="https://about.gitlab.com/">GitLab</a> has also a container registry. GitLab uses
it to manage build and it’s available for you from version 8.8 if you are
already using this tool.</p>
<p class="text-muted">Thanks <a href="https://twitter.com/kishoreyekkanti" target="_blank">Kishore
Yekkanti</a>, <a href="https://twitter.com/liuggio" target="_blank">Giulio De
Donato</a> for your review.</p>
<div class="post row">
<div class="col-md-12">
<div class="bs-callout bs-callout-info row">
<div class="row">
<div class="col-md-12">
<h2><a href="//gianarb.it/blog/docker-the-fundamentals" target="_blank">get "Docker the Fundamentals"</a> <small>by. Drive your boat as a Captain</small></h2>
</div>
</div>
<div class="row">
<div class="col-md-3">
<a href="//gianarb.it/blog/docker-the-fundamentals" target="_blank"><img src="/img/the-fundamentals.jpg" class="img-fluid" /></a>
</div>
<div class="col-md-9">
<p>
You can get the Chapter 2 of the book <a href="/blog/scaledocker" target="_blank">"Drive your boat as a Captain"</a> just leave click on the
cover and leave your email to receive a free copy.</p>
<p>This chapter is getting started with Docker Engine and the basic
concept around registry, pull, push and so on. It's a good way to start from
zero with Docker.</p>
</div>
</div>
</div>
</div>
</div>
Continuous Integration and silent checks. You are looking in the wrong placeBad and good practice when you setup a continuous integration job. Silent checks are not a good practice but analyze your code is the perfect way to understand how your codebase is evolving.https://gianarb.it/img/jenkins.png2016-11-18T10:08:27+00:002016-11-18T10:08:27+00:00https://gianarb.it/blog/continuous-integration-and-silent-checks<p>Continuous Integration is a process of merging all developer working copies to
shared mainline several times a day. In practice is when you have in place a
system that allow you to trust all changes that all developers are doing in a
short period of time in order to have that code complaint and ready to be
pushed in production.</p>
<p>There are a lot of different way to do CI but I will stay focused on a very
important expect, you need a policy that contains a series of checks that you
can easy automate. All this steps persisted on every change allow you to mark
that new code as <code>ready</code>.</p>
<p>Automation is an important part to keep your integration continuous, usually what
people do is a human review of the code, if one or more people mark your code
as complaints and the continuous integration system is agree with them your code
can be merged. This is the unique manual step.</p>
<p>But let’s talk about what I call “Silent Checks” they are really one of the
best invention that I never saw. Silent Checks are like cigarettes, all know
that they are not so good but nobody cares.</p>
<p>Usually your CI system use exit code to understand if a check is good or bad,
your command come back with <code>0</code> in case of success or with another number if
something fails. Sometime you can find in your continuous integration checks
that put the status code in a silent mode. The check fails but it’s not important enough.</p>
<p><img class="img-fluid" src="/img/the-wolf-ci.jpeg" alt="continuous integration party" /></p>
<p>You have a check that runs but you are not asking people to care about the
result. Probably because it’s not important enough. There are few disadvantages
about this approach:</p>
<ul>
<li>That check is making your job slow.</li>
<li>If the job doesn’t fail no one care about that optional check and your check
will never fail.</li>
<li>When a job fails you just need to scroll and jump over all the logs generated
by the optional check. They produce a very long logs because usually they
fails. There is more, usually your coworkers forget about this check and they
ping you about that errors.</li>
</ul>
<p>Analyse your code is very important but there are other strategies that
you can use to avoid this inconvenient. Usually the silent checks are in place
in a period of migration, maybe they are important to monitor how it is going.
They are just in the bad position.
You can move them in a separated job, collected them and analyse what you need
to analyse and monitoring trends about how your team works.</p>
<p>I saw a TEDx Talk by Adam Tornhill. He talked about Analyzing Software with
forensic psychology. This topic is great! You can get a lot of informations
about your application from who is writing that code.</p>
<div style="text-align:center">
<iframe width="640" height="360" src="https://www.youtube.com/embed/qJ_hplxTYJw" frameborder="0" allowfullscreen=""></iframe>
</div>
<p>Trends and monitoring not just to understand how your application works but
they are fundamentals to understand how your team is working, how they
feel and also to catch how your codebase is moving. They are really important
and if you are strong enough to have a good monitoring system for that metric
you are really in a good position! You just need to understand that insert
them into the continuous integration flow is not a good idea.</p>
Docker Bench SecurityContainer security is a hot topic because today containers are everywhere also in production. It means that we need to trust this technology and start to think about best practices and tools to make our container environment safe.https://gianarb.it/img/docker.png2016-11-15T10:08:27+00:002016-11-15T10:08:27+00:00https://gianarb.it/blog/Docker-Security-Benchmark<p>Frequently, best practices help you to have a safe environment,
<a href="https://github.com/docker/docker-bench-security">docker-bench-security</a> is an
open source project that runs in a container and scans your environment to
report a set of common mistakes like:</p>
<ul>
<li>Your kernel is too old</li>
<li>Your docker is not up to date</li>
<li>Some Docker daemon configurations are not good enough to run a production environment</li>
<li>Your container runs 2 processes</li>
<li>and others</li>
</ul>
<p>It’s a great idea to run it at some stage in each host to have an idea about
the status of your environment. To do that you can just use this command when
running a container</p>
<pre><code class="language-bash">$ docker run -it --net host --pid host --cap-add audit_control \
-v /var/lib:/var/lib \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /usr/lib/systemd:/usr/lib/systemd \
-v /etc:/etc --label docker_bench_security \
docker/docker-bench-security
</code></pre>
<p>A good way to start is your run it in your local environment. Run the command
and check what you can do to make your local environment safe. This tool is
open source on GitHub and it’s also a great example of collaboration and how a
community can share experiences to help other members to improve an
environment. This is a partial output:</p>
<pre><code class="language-bash">Initializing Thu Nov 24 21:35:24 GMT 2016
[INFO] 1 - Host Configuration
[WARN] 1.1 - Create a separate partition for containers
[PASS] 1.2 - Use an updated Linux Kernel
[PASS] 1.4 - Remove all non-essential services from the host - Network
[PASS] 1.5 - Keep Docker up to date
[INFO] * Using 1.13.01 which is current as of 2016-10-26
[INFO] * Check with your operating system vendor for support and security maintenance for docker
[INFO] 1.6 - Only allow trusted users to control Docker daemon
[INFO] * docker:x:999:gianarb
[WARN] 1.7 - Failed to inspect: auditctl command not found.
[WARN] 1.8 - Failed to inspect: auditctl command not found.
[WARN] 1.9 - Failed to inspect: auditctl command not found.
[INFO] 1.10 - Audit Docker files and directories - docker.service
[INFO] * File not found
[INFO] 1.11 - Audit Docker files and directories - docker.socket
[INFO] * File not found
</code></pre>
<p>Sometime to have a good result you just need to run a single command.</p>
<p>This article is part of “Drive your boat like a Captain”. It’s a book about
Docker in production, how manage a cluster of Docker Engine with Swarm and what
it means to manage a production environment today.</p>
<p>Keep in touch to receive news about the book
<a href="/blog/scaledocker">scaledocker.com</a>. If you are looking for a Docker
Getting Started you can also look on the first chapter that I released <a href="/blog/docker-the-fundamentals">Docker
The
Fundamentals</a></p>
Chef Server startup notesThis tutorial explain how to setup a chef server on digitalocean from zero. It also shows how to use it to make a provisioning of one Chef Client. Chef is one of the most used provisioning tool. DevOps tool to apply infrastructure as code.https://gianarb.it/img/chef.png2016-11-10T10:08:27+00:002016-11-10T10:08:27+00:00https://gianarb.it/blog/chef-server-startup-notes<p>I worked with different provisioning tools and configuration managers in the
last couple of years: Chef, Saltstack, Puppet, Shell, Python, Terraform.
Everything that was allowing me to make automation and describe my
infrastructure as a code.</p>
<p>I really think that this is the correct street and every companies need to stop
to persist random commands in a server:</p>
<ul>
<li>The code used to describe your infrastructure is reausable.</li>
<li>The code is a good backup and you can put it in your repository to study of
it changed and manage rollbacks.</li>
<li>Your servers become collaborative and your team can review what you do.</li>
</ul>
<p>Chef is my first configuration manager, I started to use it with Vagrant few
years ago but I never had change to deep drive into it and into the full
chef-server configuration from scratch.</p>
<p>I had this change few days ago and I am here to share some notes. I used
digitalocean to start 1 Chef Server and two nodes, during this post I am not
focused about the recipe and cookbook syntax but I will share some commands and
notes that I took during my test to start and configure a Chef Server.</p>
<p>First of all <code>doctl</code> is the command line application provided by digitalocean
to manage droplets and everything, I used that tool to start my droplets.</p>
<p>The Chef Server doesn’t run in little box, we need 2gb RAM, I tried with small
size but nothing was working, the installation process gone out of memory very
soon. Thanks Ruby.</p>
<pre><code class="language-sh">$ doctl compute droplet create chef-server \
--region ams2 --size 2gb --image 20385558 \
--access-token $DO --ssh-keys $DO_SSH
$ doctl compute droplet create n1 \
--region ams2 --size 512mb --image 20385558 \
--access-token $DO --ssh-keys $DO_SSH
</code></pre>
<p><code>$DO</code> contains my digitalocean access key and <code>$DO_SSH</code> the id of the ssh key
to log into the servers. You can leave the last one empty and you will receive
an email with the password.</p>
<p>When the process is gone you will be able to copy the ip of the chef-server and go into it.</p>
<pre><code class="language-bash">$ doctl compute droplet ls
ID Name Public IPv4 Public IPv6 Memory VCPUs Disk Region Image Status Tags
30cw4230 chef-server 2gb 1 20 ams2 Debian 8.6 x64 new
q0514230 n1 512 1 20 ams2 Debian 8.6 x64 new
</code></pre>
<p>This provisioning script installs Chef-Server from the official deb package and
also install chef-manage. Chef-manage provides a nice web interface to manage
users, cookbooks and everything is stored into the server.</p>
<pre><code class="language-bash">cd /tmp
sudo apt-get update
sudo apt-get install -y unzip curl
curl -LS https://packages.chef.io/stable/ubuntu/16.04/chef-server-core_12.9.1-1_amd64.deb -o chef-server-core_12.9.1-1_amd64.deb
sudo dpkg -i chef-server-core_12.9.1-1_amd64.deb
sudo chef-server-ctl reconfigure
</code></pre>
<p>Our server is up an running reachable over HTTPS (port 443). This configuration
is just for testing purpose. It’s not a good practice leave a Chef Server public
as we are doing. It’s a better idea to close it under a VPN for example.</p>
<p>Chef supports an authentication and authorization level based on users and
companies. We are creating a new user called <code>Gianluca Arbezzano</code> with username
<code>ga@thumpflow.com</code> and as password <code>hellociaobye</code>.
We are also create an organization and we are associating user with org.</p>
<pre><code class="language-bash">chef-server-ctl user-create gianarb Gianluca Arbezzano ga@thumpflow.com 'hellociaobye' --filename /root/gianarb_test.pem
chef-server-ctl org-create tf 'ThumpFlow' --association_user gianarb --filename /root/tf-validator.pem
chef-server-ctl org-user-add tf gianarb
</code></pre>
<p>At this point we can configure a nice UI for our Chef Server with this simple
commands:</p>
<pre><code class="language-bash">sudo chef-server-ctl install chef-manage
sudo chef-server-ctl reconfigure
sudo chef-manage-ctl reconfigure --accept-license
</code></pre>
<p>Chef-Server works with the concept of Organization and User. The organization
is a group of users that share cookbooks, rules and so on. Users can update
cookbooks and there is also a set of permission to manage access on particular
resources like:</p>
<ul>
<li>Add a new node</li>
<li>Syncronize cookbook with the server</li>
<li>add new users</li>
</ul>
<p>At this point we have one user with its own key and credential. You can come
back into the UI and use username (gianarb) and password (hellociaobye) to
login in. The key (–filename) is used to configure knife and encrypt
communication between client and server. There are 3 main actors and this
point that we need to know:</p>
<ul>
<li>Chef-Server contains all our recipes, cookbooks and it’s the brain of the cluster.</li>
<li>Nodes are all servers configurated by Chef.</li>
<li>Workstation are usually enable to syncronize, update cookbooks. For example
Jenkins or your Continuous Integration System after every new commit can push
every changes into the server.</li>
</ul>
<p>Chef Server has a HTTP api and <code>knife</code> is a CLI that provide an easy
integration for your node and workstation. With this command we are installing
knife. You can do it in your local environment, to become a workstation and into
the server. (it’s usually a god practice create a user, we are doing everything
as root right know but it’s BAD! don’t be bad!).</p>
<p>We have two certificate one is <code>gianarb_test.pem</code> and it’s identify a specific
user, we need to generate our for every workstation/member of the team and the
<code>validation_client</code> represent the organization, it could be the same across
multiple users.</p>
<pre><code class="language-bash">curl -O -L http://www.opscode.com/chef/install.sh
bash ./install.sh
</code></pre>
<p>You can copy paste the 2 keys into the local machine and run this command that
will drive your into the process to create a <code>~/.chef/knife.rb</code> file that your
cli uses to communicate with the chef server.</p>
<pre><code class="language-bash">knife configure
</code></pre>
<p>This is an example of generated knife configuration file that I did in my
server. I lose times to understand the <code>chef_server_url</code> it contains the
hostname of the server but also the `/organization/<organization_short_name>'
be careful about this or knife will come back with an HTML response in your terminal.</organization_short_name></p>
<pre><code class="language-ruby">log_level :info
log_location STDOUT
node_name 'gianarb'
client_key '/root/gianarb_test.pem'
validation_client_name 'tf-validator'
validation_key '/root/tf-validator.pem'
chef_server_url 'https://chef-server:443/organizations/tf'
syntax_check_cache_path '/root/.chef/syntax_check_cache'
cookbook_path ["/home/gianarb/git/chef-pluto/cookbooks"]
ssl_verify_mode :verify_none
</code></pre>
<p>The last 2 commands download and validate the SSH certificate because in the
default configuration the CA is unofficial and we need to force our client to
trust the cert.</p>
<pre><code class="language-bash">knife ssl fetch
knife ssl check
</code></pre>
<p>Know that we did that in our server and also in our local environment we can
clone <a href="https://github.com/gianarb/chef-pluto">chef-pluto</a> a repository that contains recipes, rules and cookbooks to
configure our node, we need to syncronize it into the server.</p>
<pre><code class="language-bash">git clone git@github.com:gianarb/chef-pluto.git /home/gianarb/git/chef-pluto/chef-pluto
cd /home/gianarb/git/chef-pluto/chef-pluto
knife update /
</code></pre>
<p>The last command update all our repository into the chef server. You can log in
into the web ui and see the <code>micro</code> cookbook and the <code>power</code> rule.</p>
<p><a href="https://github.com/gianarb/micro">micro</a> is an application that I wrote in go and it just expose the ip of the
machine. It’s a binary and the cookbook downloads and starts it, pretty
straightforward.</p>
<p>At this point we need to make a provisioning of our first node, usually is the
server that install and start the Chef Client into the node, what we can do
it’s store a private key into the server to allow chef to connect to the node.
I copied the digitalocean private key into the server (~/do), from security
point of view you can create a dedicate one. You can also use the -P option if
you are not using an ssh-key to run this example.</p>
<pre><code class="language-bash">knife bootstrap <ip-node> -N node1 --ssh-user root -r 'role[power]' -i ~/do
</code></pre>
<p>If everything it’s good you can reach the application from port <code>8000</code> into the
browser. The log is something like:</p>
<pre><code class="language-bash">$ knife bootstrap 95.85.52.211 -N testNode --ssh-user root -r 'role[power]' -i ~/do
Doing old-style registration with the validation key at /root/tf-validator.pem...
Delete your validation key in order to use your user credentials instead
Connecting to 95.85.52.211
95.85.52.211 -----> Existing Chef installation detected
95.85.52.211 Starting the first Chef Client run...
95.85.52.211 Starting Chef Client, version 12.15.19
95.85.52.211 resolving cookbooks for run list: ["micro"]
95.85.52.211 Synchronizing Cookbooks:
95.85.52.211 - micro (0.1.0)
95.85.52.211 Installing Cookbook Gems:
95.85.52.211 Compiling Cookbooks...
95.85.52.211 Converging 2 resources
95.85.52.211 Recipe: micro::default
95.85.52.211 * remote_file[Download micro] action create_if_missing (up to date)
95.85.52.211 * service[Start micro] action start
95.85.52.211 - start service service[Start micro]
95.85.52.211
95.85.52.211 Running handlers:
95.85.52.211 Running handlers complete
95.85.52.211 Chef Client finished, 1/2 resources updated in 02 seconds
</code></pre>
<p>knife started the client, syncronized cookbooks, it assigned the <code>power</code> role
at the node and run the correct recipes. Your server is ready and you can
create and delete nodes to make your infrastructure complex how much you like.</p>
<p>Chef is quite old and it’s in ruby (the first one could be a plus but the
second one no really) but it continue to be a good way to make a provisioning o
your infrastructure. Lots of people moved to Ansible but the agent that they
reject offer a very good orchestration feature that it’s something that I
usually search.</p>
<p>I worked with StalStack and it’s very nice, the syntax is easy
and it seems less expensive in terms of configuration, resources and setup but
I am not really sure about the YAML specification. I am not a ruby developer
and I don’t love the ruby syntax but in the end is a programming languages and
I am doing infrastructure as a code.</p>
Docker The fundamentalsDocker The fundamental is the second chapter of my book Scale Docker. Drive your boat like a captain. I decided to share free the second chapter of the book. It covers getting started with Docker. It's a good tutorial for people that are no idea about how container means and how docker works.https://gianarb.it/img/docker.png2016-08-25T12:08:27+00:002016-08-25T12:08:27+00:00https://gianarb.it/blog/docker-the-fundamentals<p>I am writing a book about Docker SwarmKit and how manage a production
environment for your containers.</p>
<p>The second chapter of the book is a Getting Started about Docker, it covers
basic concepts about what container means and it’s a started point to
understand the concepts expressed into the book.</p>
<h2>Drive your boat like a Captain.
<small>Docker in production</small></h2>
<p>The book is work in progress but you can find more information into the site
<a href="/blog/scaledocker">scaledocker.com</a>.</p>
<p>To receive the first chapter free leave your email and if you like your twitter account:</p>
<div class="row">
<div class="col-md-6">
<img src="/img/the-fundamentals.jpg" class="img-fluid" />
</div>
<div class="col-md-4">
<form id="get-chapter">
<div class="form-group">
<label for="exampleInputEmail1">Email address *</label>
<input type="email" class="form-control" required="required" id="email" placeholder="Email" />
</div>
<div class="form-group">
<label for="exampleInputPassword1">Twitter</label>
<input type="title" class="form-control" id="twitter" placeholder="@gianarb" pattern="^@.*" />
<p class="help-block">The first letter needs to be a @</p>
</div>
<p class="text-success get-chapter-thanks">Check your email! Thanks!</p>
<p class="text-warning get-chapter-sorry"><span class="err-text"></span>.
Please notify the error with a comment or with an email</p>
<button class="btn btn-default">Get your free copy</button>
</form>
</div>
</div>
<h2>Contents</h2>
<ol>
<li>Introduction</li>
<li>Install Docker on Ubuntu 16.04</li>
<li>Install Docker on Mac</li>
<li>Install Docker on Windows</li>
<li>Run your first HTTP application</li>
<li>Docker engine architect</li>
<li>Image and Registry</li>
<li>Docker Command Line Tool</li>
<li>Volumes and File Systems 20</li>
<li>Network and Links</li>
<li>Conclusion</li>
</ol>
<p>Enjoy your reading and leave me a feedback about the chapter!</p>
<script>
(function() {
$(".get-chapter-thanks").hide();
$(".get-chapter-sorry").hide();
var api = "https://1lkdtyxdx4.execute-api.eu-west-1.amazonaws.com/prod";
$("#get-chapter button").click(function(eve) {
eve.preventDefault()
$(".get-chapter-thanks").hide();
$(".get-chapter-sorry").hide();
var requestChapter = $.ajax({
"url": api+"/the-fundamentals",
"type": 'post',
"data": {
email: $("#email").val(),
twitter: $("#twitter").val()
},
"dataType": 'json',
"contentType": "application/json"
});
requestChapter.done(function() {
$(".get-chapter-thanks").show();
});
requestChapter.fail(function(data) {
$('.err-text').html("["+data.responseJSON.code+"]"+ data.responseJSON.text);
$(".get-chapter-sorry").show();
});
});
})();
</script>
Be smart like your healthcheckIn a distributed system environment has a simple way to know the status of your server help you do understand if it's ready to go in production. HealthCheck is simple and common but design a good one can help you do avoid strange behaviors. Docker 1.12 supports healthcheck and we in this blog I share an example of implementation.https://gianarb.it/img/docker.png2016-08-25T12:08:27+00:002016-08-25T12:08:27+00:00https://gianarb.it/blog/be-start-like-your-healthcheck<p>I am not a doctor, I am a Software Engineer and this is a tech post! You can
continue to read!</p>
<p>To monitor monolithic what we usually do is install a tool
like <a href="https://www.nagios.org/">Nagios</a> to centralize all our metrics and to
stay in touch with our infrastructure and our application. In a distributed
system with more that one services with own metrics the situation is totally
different. This about how it’s more dynamic respect a monolithic. Containers
or VM that scale up and down and that move around the network, Nagios is a good
solution to check if our new service after a deploy is safe and ready to be
attached into the production pool? I love a talk made by <a href="https://github.com/kelseyhightower">Kelsey
Hightower</a> during the Monitorama event, he
speak about healthcheck watch him to follow a <a href="https://vimeo.com/173610242">great demo</a>!</p>
<p>Healthcheck is an API that your service exposes to share it’s status, if you
make it really start it’s a good tool to understand the situation of your
service with just a call. A service could be ready or not and it’s in the best
situation to communicate its status. It’s a like a patient, you need to ask
him all what you need to make the best diagnosis and take a decision about it.</p>
<p>We can stay focused on a REST service, it exposes an API under the route
/health. The response could has two different Status Code:</p>
<ul>
<li>200 if all it’s good and you service is ready</li>
<li>500 it there is something wrong and your service is not ready</li>
</ul>
<p>To make an smart HealthCheck what do we need to check?</p>
<p>This is a real implementation:</p>
<pre><code class="language-php"><?php
echo 1;
</code></pre>
<p>It’s better that nothing but we are looking for something smart! We need to
check all dependendencies that our service has and it’s for this reason that
the service itself is the best actor because it knows what it need to be ready.
I wrote a demo service, the name is <a href="https://github.com/gianarb/micro/blob/master/handle/health.go">micro</a>, it’s in go and
the version 2 use
mysql.</p>
<pre><code class="language-go">func Health(username string, password string, addr string) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
res := healtResponse{Status: true}
httpStatus := 200
dsn := fmt.Sprintf("%s:%s@tcp(%s:3306)/micro", username, password, addr)
ddb, err := sql.Open("mysql", dsn)
if err != nil {
log.Fatal(err)
}
if err := ddb.Ping(); err != nil {
res.Status = false
res.Info = map[string]string{"database": err.Error()}
}
c, _ := json.Marshal(res)
if res.Status == false {
httpStatus = 500
}
log.Println("%s called /health", r.Host)
w.WriteHeader(httpStatus)
w.Header().Set("Content-Type", "application/json")
w.Write(c)
}
}
</code></pre>
<p>Doesn’t matter how many dependencies you service has, you need to check all of
them, databases, other services that it uses. In my case I decided to add a
key-value field, I called it <code>info</code>, it contains some information about whether
mysql is or is not not working, in order to make the debug flow easy. If the
service that you are checking has an healthcheck you are lucky! You can use
that entrypoint to know if your dependency is fine. If you are not so lucky if
you can create a wrapper or just check if you can reach the service, in my case
I just tried to connect to mysql in order to know if my network supports me! I
also using the correct database name in order to avoid edge case like “mysql is
on but the database doesn’t exist”.</p>
<p>The ecosystem supports healthchecks! Nginx looks it to know if a server is
reachable, if the health check doesn’t work for a while it just make the server
out for few times. Same for Kubernetes, Swarm and Docker. Docker provides a
library in go an <a href="https://github.com/docker/go-healthcheck">healthcheck
framework</a> that you can use in your
applications, it is also used in Docker 1.12.</p>
<p>You can describe in your Dockerfile an HealthCheck</p>
<pre><code>HEALTHCHECK CMD ./cli health
</code></pre>
<p>If the exit code is 0 Docker marks you container like healthy if it’s different like unhealthy.
Very easy and flexible, you can check your REST healthcheck in this way</p>
<pre><code>HEALTHCHECK --interval=30s --timeout=30s --retries=3 \
CMD curl -si localhost:8000/health | grep 'HTTP/1.1 200 OK' > /dev/null
</code></pre>
<p><code>--interval</code> is the timing between two healthcheck, <code>--timeout</code> is used to mark
like unhealthy a service that doesn’t come back after 30s in this case,
<code>--retries</code> is the attempts to do before make a container unhealthy.</p>
<p>HealtCheck doesn’t replace traditional monitoring system but with a lot of
instances and services has a single point to check and understand the situation
after a deploy make your like easy and your products stable.</p>
Build opinions to become wiseAs a Software Engineer you need to build your own opinions about different topics like: linux or window? Editor or IDE? Containers or VM? There are different developers just for this reason. Make, share and change your opinions is the best way to grow. Not only like developer but also like human.https://gianarb.it/img/myselfie.jpg-large2016-08-25T12:08:27+00:002016-08-25T12:08:27+00:00https://gianarb.it/blog/build-opinions-to-become-wise<p>I am Gianluca, I am 24 years old and I work as Software Engineer.</p>
<p>I like my work because there are really a lot of different kinds of Software
Engineer, mainly because you can work in pretty much every environment like
technology, food, cook, sport, fashion.</p>
<p>Also because there are a lot of stuff to do, if you like to work on a product,
you build features to make happy other people to buy a car or to read a news on
a online news paper.</p>
<p>If you like play with cables, racks, switchs you can work in a big or small
farm and design fancy datanceters.</p>
<p>But I like this work because a lot of people have opinions. There are opinions
built on top of big study or after a long experience but everyone is happy to
share them and use them to build a new product.</p>
<p>I am a good PHP developer, because I worked for a while with this language and
I tried to catch and verify good opinions from a lot of developers that come
from different parts of the world and from different experiences.</p>
<p>I am happy so share someone of them with you:</p>
<pre><code class="language-php"><?php
namespace Opinion\One;
class Good {
private $important;
public function __contruct($somethingThatIReallyNeed)
{
$this->important = $somethingThatIReallyNeed;
}
}
</code></pre>
<p>Usually I force myself (and when I can other people :P ) to inject objects from
contructor if our object (Good) can not work without them.</p>
<p>This is a good way to be sure that your object will be completed, because you can not forgot nothing!</p>
<p>I use Zend Framework from a lot of time and I remember one class
<code>ServiceLocatorAwareInterface</code>.</p>
<p>I have an opion, I hate this class.</p>
<p>If you implement this interface your service has a service locator! It’s
powerfull today that you need to finish your ticket and go away with a new
feature.
After months a lot of people improve this service and they start to use a lot
of other services without think about for what reason you built your service.
Just because it’s really simple get random services from service locator.</p>
<p>Be wise, don’t allow people to write bad code and use your constructor to inject your dependencies!!</p>
<p>I have also architecture opinions like:</p>
<p>When you think about “How can I resolve this problem?” you must start from
design pattern! They are documented and tested from a lot of developers and in
more use cases.</p>
<p>Imagine your colleague that come to you around 5pm to ask how an entire
libraries is designed, you can just reply “It’s just a SOAL client, see you
tomorrow!” and you go to play basketball.</p>
<p>Or if you are building an API, OAuth2 could be a good choice for your
anthentication service. It’s tested, there are a lot of clients and
documentations.
Your clients will be good to know that you are just using
Oauth2 and nothing strange.</p>
<p>Well, I am not here to share all my opinion in a single post, all of them are
just examples of what I mean for opinion.</p>
<p>An opinion is really important! But it’s just an opinion!</p>
<p>As a Software Engineer you need to have an opinion because every day people
will try to have opinion for you: Microservices, containers, one big
repository, golang/rust.</p>
<p>To build an opinion you need to make experience and to study it’s a big effort
but to be really wise you need also to stay ready to change your
opinion.</p>
<p>In my opinion this is the main difference between smart and wise people. I
prefer the second one!</p>
<div class="alert alert-success" role="alert">
Thanks for your review <a href="https://twitter.com/fntlnz" target="_blank">Lorenzo</a>!</div>
Watch demo about Docker 1.12 made during Docker MeetupDocker 1.12 contains a lot of news about orchestration and production. During August Docker Meetup in Dublin I presented with a demo a set of new features around this new release.https://gianarb.it/img/docker.png2016-08-24T12:08:27+00:002016-08-24T12:08:27+00:00https://gianarb.it/blog/docker-1-12-meetup-dublin<p>In August during the Docker Meetup I presented with a demo some new
features provided by Docker 1.12.</p>
<p>It’s an important release because it improves your experience with Docker
in production with an orchestration framework included into the product.</p>
<p>Docker provides a new set of commands to create a cluster of Docker
deamon and manage a production enviroment.</p>
<p>It’s something like Kubernetes, Mesos, Swarm but it is included and
built-in Docker.</p>
<p>I wrote an article about it few months ago <a href="/blog/docker-1-12-orchestration-built-in">“Docker 1.12 orchestration
built-in”</a>.</p>
<p>In this demo I do an introduction of some new features like:</p>
<div style=" text-align: center;">
<iframe width="420" height="315" src="https://www.youtube.com/embed/h7a7vhzjElo" frameborder="0" allowfullscreen=""></iframe>
</div>
<ul>
<li>How create a SwarmMode docker cluster</li>
<li>What is a service? What tasks means?</li>
<li>How Docker SwarmKit manage a node down?</li>
<li>I tried to show the HealthCheck feature :)</li>
<li>How docker swarmkit manage containers update</li>
<li>service discovery</li>
</ul>
“Microservices and common parts"When you think about microservices and distributed system there are a lot of parts that usually all your services require. Logging, monitoring, testing, distribution. Manage them in the best way it's one of the reason of success for your distributed system. In this article I shared few of this parts with some feedback to design them in a good way.https://gianarb.it/img/distributed_system_planet.png2016-08-14T12:08:27+00:002016-08-14T12:08:27+00:00https://gianarb.it/blog/services-and-common-parts<p>Changing my glossary and replacing the concept of application with
service could be a buzzword but this takes me to built a
new approach to my work.</p>
<p>Nowadays many products require more services than before to
work: perhaps they could be modules, libraries directly integrate in one or
more services or applications that communicate and provide a feature, it
doesn’t matter cause in any case the product’ll have some dependences.</p>
<p>If you start to follow this path, a lot of redundant concepts will show up in your
product:</p>
<ul>
<li>Monitoring</li>
<li>Logging</li>
<li>Authentication</li>
<li>Scaling</li>
<li>High availability</li>
<li>Distribution</li>
<li>Testing (unit, functional, integration..)</li>
<li>And others</li>
</ul>
<p>Some of them, like monitoring or logs, require architecture and tools
selection: you can use some B2B tools or host something in house. It’s not
only a problem of tooling, the other face of this “redundant money” is how your
services can communicate logs, metrics with outside in a clean and reusable
way.</p>
<p>In this post I will try to share the common part of a
microservices ecosystem and some possible approach to solve this issues.</p>
<h2 id="logging">Logging</h2>
<p>All applications require a good and strong log system. There are few
libraries able to help you in managing this section but the minimum requirement, in my opinion, includes:</p>
<ul>
<li>Support for multiple stream: usually, I use stdout or file and I move
them in a database with a separate pipeline, but a lot of good libraries allow you to manage your logs in different collectors.</li>
<li>Different layers like: INFO, DEBUG, WARNING, FATAL.</li>
<li>Provide a way to change this layer runtime, for example with a RPC call.</li>
</ul>
<p>The third point is really important: if your application start to have a big
traffic, the amount of logs you must manage will be relevant; so, changing this
level runtime allows you to manage the amount of logs that you store and, for example, allows DEBUG
information only if you need to do some specific debugging in production. This strategy save storage and money.</p>
<p>There are a lot of services and open source tools able to manage and storage this data. The real issue is decide which street follow.</p>
<p>Are you interested to manage your logs or it’s a big effort for your company? You can move all to log entries and forgot about elastic search and kibana
and similar in this case. Think about your environment and catch the best solution. Remember that it could be just a temporary solution. When you start a business you have different thoughts, start slim and easy.</p>
<h2 id="monitoring">Monitoring</h2>
<p>Several services require several time and energy to be monitored and to be
maintained alive.</p>
<p>The best way to do that is with a time series
database like prometheus, InfluxDB or other as a service solution like NewRelic
or AppDynamics.</p>
<p>The real problem is how your application can provide
metrics readable and usable from external systems. You can find a very good solution to this problem in Docker: they provide different streams and events to grab this kind of informations.</p>
<p>If you take a lot on how it manage this part
you can implement a good system in your application. A stream of events is
also a good API to allow other services to enjoy features provided from your
service.</p>
<h2 id="heahtcheck">Heahtcheck</h2>
<p>Understand with a single request if your application has all what it needs to
work is really important.</p>
<p>The microservices ecosystem contains a lot of micro applications that change and have dependencies to work. How can you understand if all your system is up and runs without spend a lot of time?</p>
<p>You can create for each service the call <code>/heath</code> that return 200 if all it’s fine and 500 if there is something that it’s not working properly.</p>
<p>During a release you can use this endpoint in order to understand if your
service is ready to be attached into the production pool.</p>
<p>In practice, if you have one service called Users that depends from MySQL and
from another service like Emailer, the health entrypoint for Users’s service
will check whether it can connect to the MySql and also you can call <code>/health</code> for
Emailer in order to check if the service is up.</p>
<p>Your orchestration and deploy framework can check after each deploy if the health is up and running and manage your release, it can revert or it doesn’t include your new release into the production pool.</p>
<h2 id="authentication">Authentication</h2>
<p>Your microservice is not public, sometime you have a set of firewall’s rules or
a strong network settings to manage the security of your environment but for other
services the authentication layer is a requirement and usually there are few
services that need to know which is the identity of the user that is persisting
an action.</p>
<p>Think about a To Do service, it need to know the identity of the user in order
to fetch the correct items.</p>
<p>For this reason this layer could be common between your services and it’s also a
critical section of your architecture because usually from it depends the security of
your application and users.</p>
<p>Oauth2 is a framework to manage authentication, I recommend it
because it has a documentation already done, it’s a standard.
You don’t reivent anything, there are a lot of libraries and use cases about it that make it solid, flexible and reusable.</p>
<h2 id="automation-and-deploy">Automation and Deploy</h2>
<p>A good layer of automation is important in every ecosystem to make your work less
bored but also to decrease chance for a human to make a mistake during a
repetitive task.</p>
<p>If you are thinking about a microservices ecosystem all this problems are
multiplied for a big number of applications.</p>
<p>Without a good layer of automation and a good deploy’s flow you will spend all
your day to put line of code in production without have time to stay focused on
new features or other business’s requests.</p>
<h2 id="documentation">Documentation</h2>
<ul>
<li>Describe the topology of your ecosystem,</li>
<li>how match microservices you have?</li>
<li>where they are and how they are distributed across your datacenters</li>
<li>Make it extensible and easy to read and update.</li>
<li>How a single service works?</li>
<li>Which APIs it expose</li>
<li>how another service can communicate with it.</li>
<li>Single dependencies for each microservices is also important to know.</li>
</ul>
<p>All common part like, logs, auth, metrics help you to have a
common documentation easy to maintain, read and implement but for each service
you must provide a specific documentation because all it’s clear today but
between few months when you worked on ten other services the situation could be
really different.</p>
<p>One of the goal about microservices is the possibility to add
and integrate them easily. Documentation is one of the goal to make this
possible and efficient.</p>
<h2 id="communication-layer">Communication Layer</h2>
<p>A lot of companies have one communication layer in the
environment, JSON and REST. It’s a good choice, easy to implement and there are
also a lot of tools to test, document and create client libraries.</p>
<p>But HTTP/REST is not the unique way to expose features out of your service, this is
really important to know.</p>
<p>There are other efficient and less expensive solution, binary protocol is
one of them.</p>
<p>For all this topics we can stay here to speech for years for this reason I have in
plan other posts to analyze some points better.</p>
<p>Please let me know if in your experience there are other common part between
your services.</p>
What Distributed System meansI will speak about service discovery, micro services, container, virtual machines, schedulers, cloud, scalability and latency I hope to have, at the end of this experience a good number of posts in order to share what I know and how I work and approach this kind of challenges.https://gianarb.it/img/distributed_system_planet.png2016-07-12T16:08:27+00:002016-07-12T16:08:27+00:00https://gianarb.it/blog/distributed-system-means<p>I choose to put my experience about distributed system in a serie of blog
posts on which I’ll cover different topics.</p>
<p>I will speak about service discovery, micro services, container, virtual
machines, schedulers, cloud, scalability and latency I hope to have, at the end
of this experience a good number of posts in order to share what I know and how
I work and approach this kind of challenges.</p>
<p>In first I will not speak about nothing new, in fact distributed system means:</p>
<blockquote>A distributed system consists of a collection of autonomous computers,
connected through a network and distribution middleware, which enables
computers to coordinate their activities and to share the resources of the
system, so that users perceive the system as a single, integrated computing
facility.
<p><a href="https://www0.cs.ucl.ac.uk/staff/ucacwxe/lectures/ds98-99/dsee3.pdf" target="_blank">Wolfgang Emmerich, 1997</a></p>
</blockquote>
<p>Internet is a distributed system, you infrastructure is usually a distributed
system if you follow the minimum requirements to make high availability for
your services.</p>
<p>In first of all I love what service means, your application is a service,
microservices is just a way to remind to people that a little application is
easy to maintain, deploy and control but the idea in my opinion is just make
something autonomous and useful for your customers. Sometimes your customer is
a human in other case could be another service provided by yourself or from a
third-party, it is not really important. It’s important that your service must
be ready to communicate with the extern.</p>
<p>Distributed your system is important to make it available, if you close your
service inside a single datacenter in a single part of the world you take the
risk to make it unavailable in case of problem in that particular area, if you
distribute your service in different location you are increasing the chances to
stay up.</p>
<p>You are also mitigating the latency around your system because you are bringing
your application near your customers and if you have a world wide traffic
that’s param is really important.</p>
<p><img alt="Internet Global Submarine map" src="/img/global-submarine-cable.jpg" class="img-fluid" /></p>
<p>This is the map of the submarine cable (2014) and all know that internet is not
in the air and serve different point in the world require different amount of
time to have a response and also is not just a problem of distance but traffic
and quality of the network have them weight. Akamai is a expert about this
topic, he provide a service of content delivery (CDN) and also it’s a
monitoring system for the status of the network, they provide different data,
one of them describe the <a href="https://www.akamai.com/us/en/solutions/intelligent-platform/visualizing-akamai/real-time-web-monitor.jsp">high level status of Internet</a>.</p>
<p>Virtualisation, container, cloud computing and in general the low price to
design an infrastructure and the growth of internet’s users allow little
company with a little budget to create something of stable, secure and
available in different part of worlds. I think that for this reason micro
services and distributed system start to have a big impact in the industry.</p>
<p>A good exercise to understand the current situation could be design a little
infrastructure cross provider in multi datacenter to support a normal blog,
with a database and an application. With a couple of servers on different cloud
providers you can create an high available and distributed system across
multiple datacenter and avoid a lot of point of failure like: Geography
disaster Provider errror…</p>
<p>Docker, openstack, AWS, Consul, Prometheus, Elasticsearch, MongoDB are just a
set of products that help us to create something really stable and useful.
Continue Delivery, High Availability, disaster recovery, monitoring, Continuous
Integration, reliable are a subset of topic that you must resolve when you
think about distributed system because you can not care about where the
instances of your applications are around the world and the network is not a
paradise of stability. Microservices helps you to create better and stable
application, allow your company to create more rooms for more developers and to
replace single pieces and features but they create other kind of problems like
architecture complexity, good knowledge of in different layers (DevOps point of
view), network and chain of failures. All the topics that we already know must
be adapter for this new architecture, monitoring, logging, deploy.</p>
Symfony and InfluxDB to monitor PHP applicationHow monitoring your Symfony and PHP applications with InfluxDB.https://gianarb.it/img/influx.jpg2016-07-02T10:08:27+00:002016-07-02T10:08:27+00:00https://gianarb.it/blog/symfony-and-influxdb-to-monitor-php-applications<p>Symfony is one of the most famous PHP Frameworks in use right now, today we are
going to use it to understand how much is important to know how one our
features performs. We don’t monitor CPU usage, I/O disk or the number of
server errors but we are monitoring the final feature from the business point
of view.
This approach is very important because understanding which is the
impact of a new release on a specific, critical feature is the best way to
understand how a customer use our service.</p>
<p>In this article we are implementing
a monitor for one of the most diffuse business requirements, the
authentication.</p>
<p>In order to understand how many people try to do a login, and
track how many of them perform a wrong authentication and use this metrics to
understand how the system evolves.
Sometimes happens that right after a deploy
the number of wrong logins grows faster than usual, this could be a sign that
the feature doesn’t work as expected. We begin from the standard Symfony
application</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$ composer create-project symfony/framework-standard-edition influxdb_app
$ cd influxdb_app/
$ php bin/console server:run</code></pre></figure>
<p>We create a route under authorization (/admin), it is private and only admin
users are allowed to see that and one public homepage (/).</p>
<p>You can follow the official tutorial, or this step from the Symfony application
directly from GitHub. We have an admin panel and a public site, the idea is
use our InfluxDB PHP SDK to understand how this feature works. We use the
Dependency Injection Container (DiC) provided by Symfony to create our
influxdb.client.</p>
<p>Go into the project’s root and use composer to install the library: composer
require influxdb/influxdb-php The first things to do is add some parameters:
host and port of our InfluxDB. To do that open app/config/parameters.yml and
add this fields:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml">influxdb_host: 127.0.0.1
influxdb_port: 8086
influxdb_db: symfony_influx</code></pre></figure>
<p>We use the REST Api to send metrics to InfluxDB, please if your connection
params are different change them.</p>
<p>The second step is configure the Symfony’s
DiC in order to get our client around the application, open
app/config/services.yml and add this line.</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml">services:
# ...
influxdb_client:
class: InfluxDB\Client
arguments: ['%influxdb_host%', '%influxdb_port%']
influxdb_database:
class: InfluxDB\Database
arguments: ['%influxdb_db%', '@influxdb_client']</code></pre></figure>
<p>With this specification we are asking at the DiC to provide a influxdb_client,
it’s a InfluxDB\Client object with two constructor parameters: influxdb_host,
influxdb_port.</p>
<p>InfluxDB could have different databases, influxdb_database is a
service that use the influxdb_clint to work with only one database influxdb_db.
Now we have a influxdb.database ready to be used! Only to try if all works
fine open DefaultController and try to send a page view metrics:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php">/**
* @Route("/", name="homepage")
*/
public function indexAction(Request $request)
{
$result = $this->get("influxdb_database")->writePoints([new Point(
'page_view', // name of the measurement
1 // the measurement value
)]);
// replace this example code with whatever you need
return $this->render('default/index.html.twig', [
'base_dir' => realpath($this->getParameter('kernel.root_dir').'/..'),
]);
}</code></pre></figure>
<p><img class="img-fluid" alt="InfluxDB admin panel" src="/img/influxdb_admin.png" /></p>
<p>Go into the homepage and in the meantime do a query like SELECT * FROM
“symfony_influx”.”“.page_view into the InfluxDB’s admin panel, you are sending
a new point after each visit! Very good but we have another target! If you
have some problem and you are using my repository see the difference between
this and the last step on GitHub.</p>
<p>Sent a point in this method it’s not a good
practice because our controller has two responsability: Rendering of the page
Sent a point In this example the situation is not dangerous because the
application is very easy and with a very low traffic, but symfony provide a
strong event system, perfect to split the logic on different classes and
simplify our code, we try to follow this approach for our last step, we create
a listener to sent a point when an user fails a login. In first we must create
a listener into src/AppBundle/Listener/MonitorAuthenticationListener.php.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
namespace AppBundle\Listener;
use Symfony\Component\Security\Core\Event\AuthenticationFailureEvent;
use InfluxDB\Point;
class MonitorAuthenticationListener
{
private $database;
public function __construct($database)
{
$this->database = $database;
}
public function onFailure(AuthenticationFailureEvent $event)
{
$this->database->writePoints([new Point(
'login',
1,
['status' => 'error']
)]);
}
}</code></pre></figure>
<p>We use the DiC to attach this listener at the security.authentication.failure
event. This event is called after each failed login. To do that open
app/config/services.yml and add this configuration.</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml">services:
# ....
security.authentication.monitoring:
class: AppBundle\Listener\MonitorAuthenticationListener
arguments: ['@influxdb_database']
tags:
- { name: kernel.event_listener, event: security.authentication.failure, method: onFailure }</code></pre></figure>
<p>We are injecting into the constructor our influxdb database, in this way we use
it to send points like our old example into the controller. This is the last
practical section of this tutorial, please if you have lost something try to
check this diff from the last step on GitHub. Try to do some wrong login and
check the situation into the Admin Panel with a query like</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql">SELECT * FROM "symfony_influx"."".login.</code></pre></figure>
<p><img class="img-fluid" alt="InfluxDB admin panel" src="/img/chronograf.png" /></p>
<p>The admin panel is not the best way to check our metrics, InfluxData provide a
great dashboard called Chronograf, try to use this metric to create a graph
specific to understand how your feature works. This post is only a getting
started to understand a good way to send metrics without connect directly your
business logic with the monitoring system, but with a real traffic this
approach is totally inefficient.</p>
<p>Send point by point increase the traffic in your network and the latency create
performance problems, telegraf is a collector that you can use to mitigate this
problem, in this way you can not send your points directly to InfluxDB but you
can use this agent installed on your server that collect and send bulk of data
for you.</p>
Docker 1.12 orchestration built-inDocker 1.12 adds different new features around orchestration, scaling and deployment, in this article I am happy to share some tests I did with this versionhttps://gianarb.it/img/docker.png2016-06-20T10:08:27+00:002016-06-20T10:08:27+00:00https://gianarb.it/blog/docker-1-12-orchestration-built-in<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Some
tests with Docker 1.12! <a href="https://t.co/budUOtMuBB">https://t.co/budUOtMuBB</a> <a href="https://twitter.com/hashtag/docker?src=hash">#docker</a> <a href="https://twitter.com/hashtag/DockerCon?src=hash">#DockerCon</a>
orchestration, swarm and services.</p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/744977855277309953">June 20,
2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>During the DockerCon 2016 docker announced Docker 1.12 release.
One of the news stories around the new version is the orchestration system built directly
inside the engine, this feature allow us to use swarm
without installing it separately from outside, it’s now a feature provided by Docker directly.</p>
<p>Now we have a new set of commands that allow us to orchestrate containers
across a cluster.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker swarm
docker node
docker service</code></pre></figure>
<p>All these commands are focused on increasing our ability to orchestrate our
containers and also join them in services.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">#!/bin/bash
# create swarm manager
docker-machine create -d virtualbox sw1
echo "sudo /etc/init.d/docker stop && \
curl https://test.docker.com/builds/Linux/x86_64/docker-1.12.0-rc2.tgz | \
tar xzf - && sudo mv docker/* /usr/local/bin && \
rm -rf docker/ && sudo /etc/init.d/docker start" | \
docker-machine ssh sw1 sh -
docker-machine ssh sw1 docker swarm init
# create another swarm node
docker-machine create -d virtualbox sw2
echo "sudo /etc/init.d/docker stop && \
curl https://test.docker.com/builds/Linux/x86_64/docker-1.12.0-rc2.tgz | \
tar xzf - && sudo mv docker/* /usr/local/bin && \
rm -rf docker/ && sudo /etc/init.d/docker start" | \
docker-machine ssh sw2 sh -
docker-machine ssh sw2 docker swarm join $(docker-machine ip sw1):2377</code></pre></figure>
<p>another Captain wrote this script that I just updated to work
with the public Docker 1.12-rc2. We can use this script to create a cluster with
virtual box ready to be used. After this script you can see the number of
workers and masters, in this case your one and one.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$ docker node ls</code></pre></figure>
<p>Docker 1.12 has a built-in set of primitive functions to orchestrate your containers just
like a summary. The main commands that you must run to create a cluster are</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">## On the master to start your cluster
$ docker swarm init --listen-addr <master-IP(this ip)>:2377
## on each node to add it into the cluster
$ docker swarm join <master-ip>:2377</code></pre></figure>
<p><img class="img-fluid" alt="Docker Swarm architecture" src="/img/posts/swarm_arch.png" /></p>
<p>If you are not confident with docker swarm this is the architecture, this graph
is provided by Docker Inc. and explains really well the design around this project.
The principal actors are managers and workers, managers are the brains of the
system, they dispatch schedules and remember services and containers. Workers
execute these commands.</p>
<p>The cluster is secure because each node has a proper TLS
identity and all communications are encrypted end to end by default with a
automatic key rotation in order to increase the security around the keys use in
the cluster.</p>
<p><a href="https://raft.github.io/">Raft</a> is the consensual protocol used to distribute
message around the cluster and check the number of nodes, it’s complex
algorithm but really interested I have in plan another article about it but the
offical site contains a lot of details about it.</p>
<p>We already saw the concept of services in docker-compose they are a single or a
group of containers to describe your ecosystem, you can scale a specific
service or orchestrate it across your cluster. It’s the same here, you don’t have
a specification file like compose at the moment but anyway you can run a bunch
of commands to create your service.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$ docker service create --name helloworld --replicas 1 alpine ping docker.com</code></pre></figure>
<p>With this example we push up a new service helloworld. It has one container from
the alpine image and it pings docker.com site.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker service ls</code></pre></figure>
<p>To watch all our services, we can also inspect a service</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker service inspect <container_id></code></pre></figure>
<p>There is a new concept, when you run a service you are also creating a task,
this task represents the container/s under your service, in this case we have
just one task</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker service tasks helloworld</code></pre></figure>
<p>When you scale your service you are creating new tasks</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker service scale helloworld=10</code></pre></figure>
<p>Now you can see 10 tasks that are running and you can inspect one of them,
inside you can find the containerId and you can, for example, follow logs</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">22:17 $ docker inspect 6fhfse4it8lwzlsk1t5sd5jbk
[
{
"ID": "6fhfse4it8lwzlsk1t5sd5jbk",
"Version": {
"Index": 67
},
"CreatedAt": "2016-06-18T21:06:36.707664178Z",
"UpdatedAt": "2016-06-18T21:06:39.241942781Z",
"Spec": {
"ContainerSpec": {
"Image": "alpine",
"Args": [
"ping",
"docker.com"
]
},
"Resources": {
"Limits": {},
"Reservations": {}
},
"RestartPolicy": {
"Condition": "any",
"MaxAttempts": 0
},
"Placement": {}
},
"ServiceID": "24e0pojscuj2irvlxvx2baiid",
"Slot": 2,
"NodeID": "55v4jjzf56mcwnhbwvn4cq1rs",
"Status": {
"Timestamp": "2016-06-18T21:06:36.7110425Z",
"State": "running",
"Message": "started",
"ContainerStatus": {
"ContainerID": "4ec69142e3e886098915140663737f4176c6de5afe9f2fad1f5b2439d8fc336d",
"PID": 3627
}
},
"DesiredState": "running"
}
]
22:17 $ docker logs -f 6fhfse4it8lwzlsk1t5sd5jbk</code></pre></figure>
<p>At this point it is a normal container and it’s running on your cluster.
Well I tried to explain the main concept around this big feature provided by
Docker 1.12, the last example is just to cover the DNS topic.</p>
<p>I created an application that serve an http server and print the current IP.
Each server has an internal load balancer that dispatches traffic in round robin
between the different tasks.
In this way it’s totally transparent, you can just
resolve your service with a normal URL, docker will do the rest for you.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$. docker service create —-name micro —-replicas 10 —-publish 8000/tcp gianarb/micro</code></pre></figure>
<p><a href="https://github.com/gianarb/micro">Micro</a> is an application that exposes an
http server on port 8000 and print the current ip, now we have 10 tasks with
this service.
To grab the current entry point for our service we can inspect it
and search for this information:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$. docker service inspect <id-service>
...
"Endpoint": {
"Spec": {},
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 8000,
"PublishedPort": 30000
}
],
"VirtualIPs": [
{
"NetworkID": "890fivvc6od3pa4rxd281lobb",
"Addr": "10.255.0.5/16"
}
]
}
...</code></pre></figure>
<p>In this case our published port is 3000, we can call <ip>:3000 to resolve our
service, if you try to do multi request you can see your IP chances because the
internal DNS is calling different containers.</ip></p>
<p>This is just an overview about the features but there are other powerful news
like DAB, stacks and how do an easy update of your containers, this could be
the topic around my next article. Please stay in touch follow me on
<a href="https://github.com/gianarb">Twitter</a> to chat and receive news about the next articles.</p>
<blockquote>
<p>Thanks <a href="https://twitter.com/gpelly">@gpelly</a> for your review!</p>
</blockquote>
A little bit of refactoringStrategy to refactoring your code. Tricks about performance from PHPKonf Istanbul Conference.https://gianarb.it/img/refactoring.jpg2016-04-24T10:08:27+00:002016-04-24T10:08:27+00:00https://gianarb.it/blog/a-little-bit-of-refactoring<p>I wrote this note during the <a href="https://phpkonf.org/">PHPKonf</a>, I spoke about
<a href="/jenkins-real-world/#/">Jenkins and Continuous Delivery</a> but
during the days I followed few interested talks and this is my list of notes.</p>
<p>Is the code reusable? For a few people the response is yes, for other ones is
no. I agree with <a href="https://twitter.com/ocramius">Ocramius</a> that the response is
no. An abstraction is reusable, an interface is reusable, but it’s very hard to
reuse a final implementation. First of all because when you finish to write it
your code is already old, your function is already legacy and you start to
maintain it, you search bugs and edge cases.</p>
<p>One of the way to reduce the time dedicated to do refactroring is prevent and
defende your code from bad integration, in OOP usually you have a sort of
visibility (private, public, protected) and other different way to defend your
code, opinable but the ocramius’s talk about <a href="https://ocramius.github.io/extremely-defensive-php/#/">Extremly defensive
php</a> is good to see.</p>
<p>Refactoring is a methodology to make better your code. There are different
improvement topics like readable, performance, solidity.</p>
<ul>
<li>make your code readable for the new generation is one of the best stuff that
you can do to show your love for your team and your company.</li>
<li>If your site require more time to be loaded usually you lost your client.
“less performance is a bug” cit. <a href="https://twitter.com/fabpot">Fabien Potencier</a></li>
<li>When your run your code all it’s fine, you are a good developer and your
feature works. After the deploy in production your code is the same but
usually there is a bd category of people, your client, that will use it in a
very strange way, usually it’s synonym of bug or edge case. Each bug fix make
your code more solid.</li>
</ul>
<p>Test your code before start to change it, you know automation is good but if
you love seems a machine to it manually. Setup a continuous integration
system, it can be do just one step like run tests but remember to increase that
with all steps that you usually do to test the compliance of your code like
style, standard, static analysis just to enforce that you are not a machine.
Create a good environment and an automatic lifecycle for your application allow
you to stay focused on the code and not lost your time around stupid task,
remember that when a routine is good the machine fail less respect a human,
usually.</p>
<p>Refactoring is one of the best stuff that you can do for other people and to
make your feature ready for the real world, usually it’s hard for no-tech
company understands it because few times they don’t see any kind of change
create a good environment to save time and use it to do refactoring is a godo
strategy. Automation is the unique method that I know to do that. There are
different layer of automation just to start my 2coins is just put a make file
on your codebase and when you do something for the second time stop to write it
on your console and write a new make task to share with you team. After that
install Jenkins and allow it do run this task for you before put on your on the
master branch (for git users, trunk for svn users).</p>
<p>Make your development environment comfortable and increase the conformability’s
perception about the lifecycle it’s the best way to do refactoring without the
fear to die. If you are fear to die usually you don’t do nothing.</p>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Great
talk as always Gianluca :) <a href="https://twitter.com/GianArb">@GianArb</a>
<a href="https://twitter.com/hashtag/phpkonf?src=hash">#phpkonf</a> <a href="https://t.co/ZW2G1UsXm7">pic.twitter.com/ZW2G1UsXm7</a></p>—
Fontana Lorenzo (@fntlnz) <a href="https://twitter.com/fntlnz/status/733986655334486016">May 21,
2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Add the PHPKonf in your list! See you next year!</p>
Docker inside docker and overview about Jenkins 2A little overview about Jenkins 2 but the main topic of the article is about how run docker inside docker to start a continuous integration system inside a containerhttps://gianarb.it/img/docker.png2016-04-01T10:08:27+00:002016-04-01T10:08:27+00:00https://gianarb.it/blog/docker-inside-docker-and-jenkins-2<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr"><a href="https://twitter.com/hashtag/docker?src=hash">#docker</a> inside docker
and an overview about Jenkins 2 <a href="https://t.co/qa5ddjfhrs">https://t.co/qa5ddjfhrs</a> <a href="https://twitter.com/docker">@docker</a> <a href="https://twitter.com/jenkinsci">@jenkinsci</a> <a href="https://twitter.com/hashtag/container?src=hash">#container</a></p>—
Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/727876226875068416">May 4,
2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Jenkins is one of the most famous
continuous integration and deployment tools, it’s written in Java and it helps
you to manage your pipeline and all tasks that help you to put your code in
production or manage your build.</p>
<p>The announcement of Jenkins release of version 2 few days ago, is one of the
best release of this year in my opinion.</p>
<p>The previous version is very stable but it has a lot of years and the ecosystem
is totally different. I am happy to see a strong refurbishment to get the best
of this powerful tool with a series of new feature like:</p>
<ul>
<li>Nice installation wizard</li>
<li>Refactoring of the design, one of the most critical
feature of the previous version</li>
<li>Good and modern set of plugins like <a href="https://jenkins.io/solutions/pipeline/">Jenkins
Pipeline</a> to manage your build</li>
</ul>
<p>Jenkins is truly a wonder but the tool of the moment it’s docker, engine
that allow you to work easier with the containers.</p>
<p>This two tools together are perfect to create an isolated environment to test
and deploy your applications.</p>
<p>The first setup could be install Jenkins on your
server and use a plugin to manage the integration and trigger your test inside
an isolated environment, the container.</p>
<p>Great work but in my opinion reproducibility is one of the critical point when
you deal with plugins if you can not run your build on your local environment
easily then you have a problem. Secondly if the container could be a good
solution to deploy and maintain a solid and isolated application, why your
Jenkins has not the privilege to run inside a container? In this perspective
how can we run container inside a container?</p>
<p>Ok, now its the time to figure it out how to solve the problems.</p>
<p>We can use the official Jenkins image to put jenkins inside a container, but I
worked on my personal alpine installation, light and easy, <a href="https://github.com/gianarb/dockerfile/blob/master/jenkins/2.0/Dockerfile">here is the
dockerfile</a>
and we can pull it:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker pull gianarb/jenkins:2.0</code></pre></figure>
<p>If you are interested the main article to understand how run docker inside
docker is written by
<a href="https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/">jpetazzo</a>,
the idea is run our jenkins container with <code>-privileged</code> enabled and share our
docker binary and the socket <code>/var/run/docker.sock</code> to manage our
communications.</p>
<ul>
<li><code>/var/run/docker.sock</code> is the entrypoint of the docker daemon</li>
<li><code>docker</code> the command is like a client that sends commands to socket</li>
<li><code>--privileged</code> give extended privileges to our container</li>
</ul>
<p>Translated in a docker command:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker run -v /var/run/docker.sock:/var/run/docker.sock \
-v $(which docker):/usr/local/bin/docker \
-p 5000:5000 -p 8080:8080 \
-v /data/jenkins:/var/jenkins \
--privileged \
--restart always \
gianarb/jenkins:2.0</code></pre></figure>
<p>We connect on <code>http://docker-ip:8080</code> and start the new awesome wizard!</p>
<p><img class="img-fluid" alt="First Jenkins 2 page, grab from the log your key and start" src="/img/docker-in-docker/jenkins2-start.png" /></p>
<p><img class="img-fluid" alt="Jenkins's plugins wizard" src="/img/docker-in-docker/jenkins2-plugin.png" /></p>
<p>To verify that all work we can create a new job it only runs <code>docker ps -a</code> our
expectation is the same list of containers that we have out of jenkins.</p>
<p><img class="img-fluid" alt="Result of the first build" src="/img/docker-in-docker/jenkins2-result.png" /></p>
<p>Now we can use run command from jenkins to manage our build with docker without
any kind of plugins but anyway you are free to use <a href="https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin">Docker
Plugin</a> to start
your build.</p>
<p>I used Jenkins like an example to run docker inside another container but you
can use the same strategy to do the same with your applications if they require
a strong connection with docker.</p>
Happy docker's birthday and thanksJust a post to say thanks docker for your awesome community and happy birthday!https://gianarb.it/img/dockerbday.png2016-03-25T10:08:27+00:002016-03-25T10:08:27+00:00https://gianarb.it/blog/happy-docker-bday-and-thanks<p>Just a post to say thanks docker for your awesome community and happy birthday!</p>
<p>This week is the “Docker’s birthday week” and already it this amazing, one week
of birthday, a lot of MeetUp groups this week done a Tutorial Meetup to help
people to start with Docker, Dublin made it very well!</p>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Fantastic turnout for the <a href="https://twitter.com/hashtag/dockerbday?src=hash">#dockerbday</a> <a href="https://twitter.com/Workday">@workday</a>. Thanks to everyone who
attended and completed the voting app!! <a href="https://t.co/zWQssvyHSd">pic.twitter.com/zWQssvyHSd</a></p>—
TomWillFixIT (@tomwillfixit) <a href="https://twitter.com/tomwillfixit/status/712749765151297537">March 23,
2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>50 people to understand as docker and all ecosystem works and to eat a slice of cake (thanks WorkDay)</p>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">All
it's ready.. We can go! <a href="https://twitter.com/hashtag/dockerbday?src=hash">#dockerbday</a> Dublin..
Some problem? Don't worry to ask! <a href="https://t.co/9qz3V9mW9y">pic.twitter.com/9qz3V9mW9y</a></p>—
Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/712705450786099200">March 23,
2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>There are different kind of developers, I am happy to work and to follow each
communities that provide a good tools and increase the quality of my work, I
spend much time accross different team like doctrine, InfluxDB and I am very
happy to see as Docker make a big effort to involve and to use its community.</p>
<p>I wase member of the beautiful mentor team (we will share a “retro pic” next month
because we forget to do it) and I am happy to see that we done a good work.</p>
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr"><a href="https://twitter.com/GianArb">@GianArb</a> tnks 4 the help 2nite
Gianluca!! Appreciate it <a href="https://twitter.com/hashtag/DublinDocker?src=hash">#DublinDocker</a> <a href="https://twitter.com/hashtag/dockerbday?src=hash">#dockerbday</a></p>—
Delpedro (@Delpedro47) <a href="https://twitter.com/Delpedro47/status/712745923848351744">March 23,
2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Today we seen that also che community appreciate it!
Happy Birthday and <a href="https://www.meetup.com/Docker-Dublin/">see you next month</a>!</p>
Some days of work vs Jenkins CII love Jenkins CI is a beautiful and stable project to run job and manage continuous integration and deploy pipeline, few days ago I worked to improve the delivery pipeline in CurrencyFair and I started to do some thought about this topic, here my internal battle vs Jenkins CIhttps://gianarb.it/img/jenkins.png2016-02-21T10:08:27+00:002016-02-21T10:08:27+00:00https://gianarb.it/blog/some-days-of-work-vs-jenkins-ci<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr"><a href="https://t.co/02HbnkzRsS">https://t.co/02HbnkzRsS</a> "Some Days of work vs <a href="https://twitter.com/hashtag/JenkinsCI?src=hash">#JenkinsCI</a>" Little things about continuous integration <a href="https://twitter.com/hashtag/ci?src=hash">#ci</a> <a href="https://twitter.com/hashtag/dev?src=hash">#dev</a></p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/709466156453732352">March 14, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Guys please move down your hands, I love JenkinsCI! I am not here to write a
bad post about it!
I am here to share few days of reasonings about continuous
inteagrations, Jenkins CI and all this strong topic.</p>
<h2 id="reproducible">Reproducible</h2>
<p>There are a lot of tools that you can use to run tasks,
ant, make, grunt. Use it to run section or all your build on your local
environment, this appraoch increase the value your tasks becuase you use and
test them more times.
To have a reproducible build help you to maintain splitted your flow by your
runner maybe Jenkins is perfect but the are other tools and services:
Travis-CI, CircleCI, Drone, don’t create a big dependency with your environment.</p>
<h2 id="speedy">Speedy</h2>
<p>A slow test suite is a bad idea, in 1 minutes I can maintain the
focus on the execution but 5 minutes are a lot, you can take a coffé or
start to think about another task and return on your old task require an
effort. This several focus switch it’s not good, and at the same time you
lost 5 minutes for each build and for every engineer after 1 week are a lot
of money.</p>
<h2 id="versionable">Versionable</h2>
<p>I lost more time about this point and I am not sure if it is a required point
or not but TravisCI for example use a yml specification file, this file doesn’t
describe only your build but it part of the story of your application, if you
include it into the VCS. Could be it a value for your pipeline?</p>
<h2 id="maintainable">Maintainable</h2>
<p>There are a lot of tools that you can choice to create the perfect pipeline ant
it’s very easy lost your focus and start to use too much tools, you must try
all but it’s your task to create the perfect sub-set of tools the point 1
(Reusability) increase the value, use tools that you can reuse during the daily
work of your team to increase the develop and go the flow better.
Each tools that you add seems perfect until they don’t becase a problem.</p>
<h2 id="scalable">Scalable</h2>
<p>An easy way to decrease the time for your job is split it in different little
jobs and run them in parallel, you can check the codestyle and run your test
suite in the same time for example.
Another good reason to create a scalable environment for your jobs is because
your company would grow and the continuous integration system burns to helps it
to grow and not to stop it.</p>
<h2 id="unique">Unique</h2>
<p>Jenkins, vagrant, ant, make, drone, docker are only a list of amazing tools to
create the perfect pipeline to deploy and test your code but they are only a
means the goal is indeed the best pipeline for your code and for your team.
Observe how your team works, which the requirments and criticalities and design
the best pipeline for your use case.</p>
<h2 id="communication-layer">Communication layer</h2>
<p>One goal for your team is understand the status of the build without logged in
any application, because enter into the Jenkins site (at first because it is
not beautiful :P ) and it is another step to do other: create feature branch,
submit pull request, write code lalala..
Use directly the pull request to create a connection with your job, you
continuous integration system can submit a new comment or if you are working
with GitHub you can use the status check, in this way you can help your
colleagues during them work and remove a jump.</p>
<p>With JenkinsCi you can do all but if you lost more time to create your best
pipeline? Maybe you don’t know it or maybe it is not the best tool for your use
case. Jenkins is flexible, but the flexibility is only the number of plugins
that you can install?</p>
<p>I don’t know I use it but I am happy to experiment and there are a lot of new
technologies and tools that maybe can help us to do a good work, with or
without JenkinsCI.</p>
<h2 id="as-a-microservices">As a microservices</h2>
<p><img src="/img/pipeline.svg" alt="Continuous Integration and Deploy pipeline" /></p>
<p>This is a summary of a pipeline, each pipeline follows this steps and from this
point of view seems very easy!
Jenkins, drone as very stong solution but they are all in one, if you follow
this image it’s clear that maybe to create the own pipeline for your projects
play with the LEGO to mount the best steps for your team and for your project
it’s possible.</p>
<p>I am happy to share some projects to implement this approach.</p>
<h2 id="slimmer-proof-of-concept">Slimmer, proof of concept</h2>
<p>I tried to create a runner for my test suite, <a href="https://github.com/gianarb/slimmer">slimmer</a>,
to implement this thought with docker and go.
Go offers a lot of libraries and tools to create something in a bit of time and
docker it’s perfect because it creates isolated environment and it’s very easy
to scale with Swarm.
In practice at the moment this console app exec a <code>build.slimmer</code> a bash script
executable flexible and versionable.
<a href="https://travis-ci.org">TravisCI</a> is powerful but the YML file is it a good way
to describe a build? It’s flexible? Maybe yes but I am curious to try a “low
level” approach, because finally all becames a series of commands.
I created also a series of agent to trigger notification quicly:
<a href="https://github.com/gianarb/ircer">ircer</a>,
<a href="https://github.com/gianarb/slacker">slacker</a>. You can use them to notify the
result of your build.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">composer install
vendor/bin/phpunit
RESULT=$?
curl -LSs https://github.com/gianarb/ircer/releases/download/0.1.0/ircer_0.1.0_linux_386 > ircer
chmod 755 ircer
if [ $RESULT = 0 ]; then
./ircer -j tech-team -m "You are a great develop. Your build works"
else
./ircer -j tech-team -m "No bad but your build doesn't work"
fi</code></pre></figure>
<p>This is an example of <code>build.slimmer</code> with an IRC notification, it is a PoC and
I prepared a little <a href="/slimmer-poc-slide/#/">presentation</a> to
receive some feedback and I presented it during a Dublin Go Meetup</p>
<div class="row">
<div class="col-md-12 text-center">
<iframe width="560" height="315" src="https://www.youtube.com/embed/CWCHT3GClMM" frameborder="0" allowfullscreen=""></iframe>
</div>
</div>
<p>I wait some feedback if you are interested about continuous integration and
continuous delivery.</p>
ChatOps create your IRC bot in GoChatOps is a strong topic it is growing day by day because now with the IaaS are allowed a new way to manage your infrastacture provide for you an API layer. You can implement it to create your automation layer. A pretty bot is a good assistencehttps://gianarb.it/img/go.png2016-02-21T10:08:27+00:002016-02-21T10:08:27+00:00https://gianarb.it/blog/chatops-create-your-own-irc-bot-in-go<p>The infrastructure as a service (IaaS) opened new ways to manage your
infrastructure.
Use an API to create, destroy and update your virtual machine
is one of the biggest revolutions of our sector.</p>
<p>A lot of companies and a lot of DevOps started to create own assistence to
increase the automation or to check the status of them infrastructure, in top of
all GitHub provided a series of awesome blogpost and tools to describe this
approach that it has a name: ChatOps.</p>
<ul>
<li><a href="https://hubot.github.com/">HuBot</a> is a beautiful tools written in node.js to provide smart bot.</li>
<li><a href="https://www.pagerduty.com/blog/what-is-chatops/">So, What is ChatOps? And How do I Get Started?</a> by PagerDuty</li>
<li><a href="https://github.com/blog/968-say-hello-to-hubot">Say Hello to Hubot</a> by GitHub</li>
</ul>
<div class="row">
<div class="col-md-12 text-center">
<iframe width="560" height="315" src="https://www.youtube.com/embed/IhzxnY7FIvg" frameborder="0" allowfullscreen=""></iframe>
</div>
</div>
<p>IRC is an application layer protocol that facilitates communication. One of the
most famouse open IRC server is freenode all most important open source projects
use it to chat.</p>
<p>This concept is already applyed it because most projects are your personal bot,
for example Zend use Zend\Bot a good assistence written by DASPRiD.</p>
<p>The ChatOps is an assistence oriented to decrease the distance between your
infrastacture and your communication channels.</p>
<p>I wrote a low level library to communicate on IRC protocol, we can try to use it to
write our dummy bot.</p>
<figure class="highlight"><pre><code class="language-go" data-lang="go">package main
import (
"log"
"fmt"
"regexp"
"bufio"
"net/textproto"
"github.com/gianarb/go-irc"
)
func main(){
secretary := NewBot(
"irc.freenode.net",
"6667",
"SybilBot",
"SybilBot",
"#channel-name",
"",
)
conn, _ := secretary.Connect()
defer conn.Close()
reader := bufio.NewReader(bot.conn)
tp := textproto.NewReader(reader)
for {
line, err := tp.ReadLine()
if err != nil {
log.Fatal("unable to connect to IRC server ", err)
}
isPing, _ := regexp.MatchString("PING", line)
if isPing == true {
bot.Send("PONG");
}
fmt.Printf("%s\n", line)
}
}</code></pre></figure>
<p>With this code you have a bot, in this case her name is SybilBot and at the
moment it suppot only the PING PONG flow, without this helth system your bot go
down after few time.</p>
<p>You can use the same log to add other actions</p>
<figure class="highlight"><pre><code class="language-go" data-lang="go">yourAction, _ := regexp.MatchString("CheckSomething", line)
if yourAction == true {
// Do Something
}</code></pre></figure>
<p><a href="https://github.com/gianarb/go-irc">go-irc</a> allow you to communicate over IRC protocol, our but is very stupid I
like the idea! If you are working on this topic, in go or in other language
please ping me! I am very happy to know your bot!</p>
InfluxDB PHP 1.3.0 is ready to goInfluxDB is a time series database, it helps us to manage matrics, point and offert a stack of tools to collect and see this type of data. I am a maintainer of InfluxDB PHP integration. In this past I describe the news provided by new relesae 1.3.0https://gianarb.it/img/influx.jpg2016-02-18T10:08:27+00:002016-02-18T10:08:27+00:00https://gianarb.it/blog/influxdb-php-release-1-3-0<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Shout out to <a href="https://twitter.com/GianArb">@GianArb</a> for shipping a new release of the InfluxDB-PHP library! Here's what's new: <a href="https://t.co/tJQIu9OCbL">https://t.co/tJQIu9OCbL</a></p>— InfluxData (@InfluxDB) <a href="https://twitter.com/InfluxDB/status/704403294592970752">February 29, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>We are happy to annuonce a new minor release, <a href="https://github.com/influxdata/influxdb-php">Influxdb-php library</a> 1.3.0.</p>
<p>This is a list of PRs merged in 1.3.0 since 1.2.2:</p>
<ul>
<li><a href="https://github.com/influxdata/influxdb-php/pull/36">#36</a> Added quoting of dbname in queries</li>
<li><a href="https://github.com/influxdata/influxdb-php/pull/35">#35</a> Added orderBy to query builder</li>
<li><a href="https://github.com/influxdata/influxdb-php/pull/37">#37</a> Fixed wrong orderby tests</li>
<li><a href="https://github.com/influxdata/influxdb-php/pull/38">#38</a> Travis container-infra and php 7</li>
</ul>
<p>The <code>QueryBuilder</code> now support the orderBy function to order our data, InfluxDB supports it from version 0.9.4.</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql">select * from cpu order by value desc</code></pre></figure>
<p>Now you can do it in PHP</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php">$this->database->getQueryBuilder()
->from('cpu')
->orderBy('value', 'DESC')->getQuery();</code></pre></figure>
<p>We are increase our Continuous Integration system in order to check our code with PHP7, it’s perfect!</p>
<p>We escape our query to support reserved keyword like <code>database</code>, <code>servers</code> personally I prefer avoid this type of word but you are free to use them.</p>
<p>Please we are very happy to understand as the PHP community use this library and InfluxDB, please share your experience and your problem into the repository, on IRC (join influxdb on freenode) and we wait you on <a href="https://twitter.com/influxdata">Twitter</a>.</p>
<p>Remeber to update your <code>composer.json</code>!</p>
<p>```json</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">{
"require": {
"influxdb/influxdb-php": "~1.3"
}
}</code></pre></figure>
<p>A big thanks at all our contributors!</p>
Swarm scales docker for freeDocker is an awesome tool to manage your container. Swarm helps you to scale your containers on more servers.https://gianarb.it/img/docker.png2015-12-14T10:08:27+00:002015-12-14T10:08:27+00:00https://gianarb.it/blog/swarm-scales-your-containter-for-free<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">An ocean of containers! With docker and swarm.. <a href="https://t.co/1dXoZYS3ZA">https://t.co/1dXoZYS3ZA</a> <a href="https://twitter.com/hashtag/docker?src=hash">#docker</a></p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/696620821931036672">February 8, 2016</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p><a href="https://github.com/gianarb/gourmet">Gourmet</a> is a work in progress application
that allow you to execute little applications on an isolated environment, it
dowloads your manifest and runs it in a container.
I start this application to improve my go knowledge and to work with the Docker API
I am happy to share my idea and my tests with Swam an easy way to scale this type of application.</p>
<p>Gourmet exposes an HTTP API available at the <code>/project</code> endpoint that accept a JSON request body like:</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">{
"img": "gourmet/php",
"source": "https://ramdom-your-source.net/gourmet.zip",
"env": [
"AWS_KEY=EXAMPLE",
"AWS_SECRET=",
"AWS_QUEUE=https://sqs.eu-west-1.amazonaws.com/test"
]
}</code></pre></figure>
<ul>
<li><code>img</code> is the started point docker image</li>
<li><code>source</code> is your script</li>
<li><code>env</code> is a list of environment variables that you can use on your script</li>
</ul>
<p>During my test I use this <a href="https://github.com/gianarb/gourmet-php-example">php script</a> that send a message on SQS.</p>
<p>Your script has a console entrypoint executables in this path <code>/bin/console</code> and
gourmet uses it to run your program.</p>
<p>To integrate it with Docker I used <code>fsouza/go-dockerclient</code> an open source
library written in go.</p>
<figure class="highlight"><pre><code class="language-go" data-lang="go">container, err := dr.Docker.CreateContainer(docker.CreateContainerOptions{
"",
&docker.Config{
Image: img,
Cmd: []string{"sleep", "1000"},
WorkingDir: "/tmp",
AttachStdout: false,
AttachStderr: false,
Env: envVars,
},
nil,
})</code></pre></figure>
<p>This is a snippet that can be used to create a new container.
With the container started I use the exec feature to
extract your source and to run it.</p>
<figure class="highlight"><pre><code class="language-go" data-lang="go">exec, err := dr.Docker.CreateExec(docker.CreateExecOptions{
Container: containerId,
AttachStdin: true,
AttachStdout: true,
AttachStderr: true,
Tty: false,
Cmd: command,
})
if err != nil {
return err;
}
err = dr.Docker.StartExec(exec.ID, docker.StartExecOptions{
Detach: false,
Tty: false,
RawTerminal: true,
OutputStream: dr.Stream,
ErrorStream: dr.Stream,
})</code></pre></figure>
<p>After each build Gourmet cleans all and destroies the environment.</p>
<figure class="highlight"><pre><code class="language-go" data-lang="go">err := dr.Docker.KillContainer(docker.KillContainerOptions{ID: containerId})
err = dr.Docker.RemoveContainer(docker.RemoveContainerOptions{ID: containerId, RemoveVolumes: true})
if(err != nil) {
return err;
}
return nil</code></pre></figure>
<p>At the moment it is gourmet, It could be different hypothetical use cases:</p>
<ul>
<li>high separated task</li>
<li>run a testsuite</li>
<li>dispatch specific functions</li>
</ul>
<p>A microservice to work with docker container easily.</p>
<p>I thought about an easy way to scale this application and I found
<a href="https://docs.docker.com/swarm/">Swarm</a>, it is a native cluster for docker and
it seems awesome in first because it is compatibile with the docker api.</p>
<h2 id="swarm">Swarm</h2>
<p>A Docker Swarm’s cluster is very easy to setup, I worked on this project
<a href="https://github.com/gianarb/vagrant-swarm">vagrant-swarm</a> to create a local
environment but <a href="https://docs.docker.com/swarm/install-manual/">the official
documentation</a> is easy to follow.</p>
<p>Swarm’s cluster has two actors:</p>
<ul>
<li>A master is the entrypoint of your requests, it provide an HTTP
api compatible with docker.</li>
<li>A series of nodes that communicate with the master.</li>
</ul>
<p>During this example we will work with 1 master and 2 nodes.
Build this machine with virtualbox , with another tool, or in cloud is not a
problem and <a href="https://docs.docker.com/engine/installation/">install docker</a>.</p>
<p>Into the master pull swarm and create a cluster identifier.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker pull swarm
docker run --rm swarm create
docker run --name swarm_master -d -p <manager_port>:2375 swarm manage token://<cluster_id></code></pre></figure>
<p><code>swarm create</code> returns a cluster_id use them to start the manager and the
<code>manager_ip</code> is the ip of your master server.</p>
<p>Now go into the node, because we must do few things.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
docker run -d swarm join --addr=<node_ip:2375> token://<cluster_id></code></pre></figure>
<p>When <code>cluster_id</code> is the id created in the previous step and the <code>node_id</code> is the ip
of your current node.
Enter into the master and restart your manager container</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker restart swarm_master</code></pre></figure>
<p>Now we are ready to test if all it’s up.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker -H tcp://0.0.0.0:2375 info</code></pre></figure>
<p>Replace <code>0.0.0.0.0</code> with your master ip if you are in the same server.
You’ll wait this type of response</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$. sudo docker -H tcp://192.168.13.1:2375 info
Containers: 1
Images: 1
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 2
vagrant-ubuntu-vivid-64: 192.168.13.101:2375
└ Status: Healthy
└ Containers: 1
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 513.5 MiB
└ Labels: executiondriver=native-0.2, kernelversion=3.19.0-43-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
vagrant-ubuntu-vivid-64: 192.168.13.102:2375
└ Status: Healthy
└ Containers: 0
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 513.5 MiB
└ Labels: executiondriver=native-0.2, kernelversion=3.19.0-43-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
CPUs: 1
Total Memory: 513.5 MiB
Name: f5e23167339e</code></pre></figure>
<p>Gourmet is a set of environment variables to create a connection with docker
api, in particular this function
<a href="https://godoc.org/github.com/fsouza/go-dockerclient#NewClientFromEnv">NewClientFromEnv</a>
and the <code>DOCKER_HOST</code> parameter.</p>
<p>Docker Swarm supports the same Docker API in this way gourmet uses more nodes.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$ DOCKER_HOST="tcp://192.168.13.1:2333" ./gourmet api</code></pre></figure>
Docker and wordpress for a better worldDocker and wordpress to guarantee scalability, flexibilty and isolation. A lot of webagencies install all wordpress in the same server but how can they manage a disaster? AWS with Elastic Container Service could be a more professional solution.https://gianarb.it/img/docker.png2015-12-14T10:08:27+00:002015-12-14T10:08:27+00:00https://gianarb.it/blog/wordpress-docker<blockquote class="twitter-tweet tw-align-center" lang="en"><p lang="en" dir="ltr"><a href="https://twitter.com/hashtag/docker?src=hash">#docker</a> and <a href="https://twitter.com/hashtag/wordpress?src=hash">#wordpress</a> for a better world.. <a href="https://t.co/o9c6YXvsl3">https://t.co/o9c6YXvsl3</a> Blogpost after my talk <a href="https://twitter.com/CodemotionIT">@CodemotionIT</a> How and Why? <a href="https://twitter.com/awscloud">@awscloud</a></p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/679241680797700096">December 22, 2015</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>I am trying to represent a typical wordpress infrastructure</p>
<p><img src="/img/posts/2015-12-16/wp-infra.png" alt="Wordpress typical infrastructure" /></p>
<p><strong>Isolation</strong>: every single wordpress share all with the others, filesystem,
memory, database.</p>
<p>This lack of isolation causes different problems:</p>
<ul>
<li>The monitoring of each installation is harder.</li>
<li>We share security problems</li>
<li>We don’t have the freedom to work without the fear or blocking 100 customers</li>
</ul>
<p>We are overwhelmed by the problems</p>
<p><img src="/img/posts/2015-12-16/problem.png" alt="Problem" /></p>
<h2 id="lxc-container">LXC Container</h2>
<blockquote>
<p>it is an operating-system-level virtualization environment for running multiple
isolated Linux systems (containers) on a single Linux control host.</p>
<p>by wikipedia</p>
</blockquote>
<p>Wikipedia helps me to resolve one problem (theory), container is <strong>isolated
Linux System</strong></p>
<h2 id="docker">Docker</h2>
<p>Docker borns as wrap of LXC container but now we use an own implementation
<a href="https://github.com/opencontainers/runc">runc</a> to serve your application ready
to go in an isolate environment, with own filesystem and dependencies.</p>
<p>Worpdress in this implemetation has two containers, one to provide apache and
php and one for mysql database. This is an example of Dockerfile, it describes
how a docker container works it is very simple to understand, from this example
there are different keywords</p>
<ul>
<li><code>FROM</code> describes the image that we use as start point.</li>
<li><code>RUN</code> run a command.</li>
<li><code>EXPOSE</code> describes ports to open during a link, in this case MySql runs on
the default port 3306.</li>
<li><code>CMD</code> is the default command used during the run console command.</li>
</ul>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">FROM ubuntu
RUN dpkg-divert --local --rename --add /sbin/initctl
RUN ln -s /bin/true /sbin/initctl
RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
RUN apt-get update
RUN apt-get -y install mysql-server
EXPOSE 3306
CMD ["/usr/bin/mysqld_safe"]</code></pre></figure>
<p>Very easy to read, it is a list of commands!
We are only write a container definition, now we can build it!</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker build -t gianarb/mysql .</code></pre></figure>
<p>In order to increase the value of this article and to use stable images I will
use the official <a href="https://hub.docker.com/_/mysql/">mysql</a> and
<a href="https://hub.docker.com/_/wordpress/">wordpress</a> images.</p>
<p>Download this images</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker pull wordpress
docker pull mysql</code></pre></figure>
<p>We are ready to run all! Dockerfile is only a way to describe each single
container, and the pull command downloads online container ready to work, it is
a good way to reuse your or other containers.</p>
<p>We downloaded mysql and wordpress, with the run command we start them and we
define our connections</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker run \
--name mysql \
-p 3306:3306 \
-e MYSQL_ROOT_PASSWORD=passwd mysql
docker run -e WORDPRESS_DB_HOST=wp1.database.prod \
-e WORDPRESS_DB_USER=root \
-e WORDPRESS_DB_PASSWORD=help_me \
-p 8080:80 \
-d --name wp1 \
--link wp.database.prod:mysql wordpress</code></pre></figure>
<p>I can try to explain this commands, it run two containers:</p>
<ul>
<li>The name of the first container is mysql and it uses the <code>mysql</code> image, we
use -p flag to expose mysql port now you can use phpmyadmin or other client
to fetch the data but remember that is not a good practice.</li>
<li>The second container called wp1 uses the image <code>gianarb/wordpress</code> forward
the container port 80 (apache) on host 8080, that in this case it is the way
to see the site. –link flag is the correct way to consume mysql outside the
main container, in this particular case we could use wp.database.prod how url
to connect at mysql from our worpdress container, awesome!</li>
<li>Docker image supports environment variable <code>ENV</code> for example we can use them
to configure our services, in this case to set root password in mysql and to
configure worpdress’s database connection</li>
</ul>
<p>We are ready! Now you have a worpdress ready to go on port 8080.</p>
<h2 id="docker-compose">Docker Compose</h2>
<p>To save time and to increase reusability we can use
<a href="https://docs.docker.com/compose/">docker-compose</a> tool
that helps us to manage multi-container infrastructures, in this case one for
mysql and one for wordpress.
In practice we can describe all work did above in a <code>docker-compose.yml</code> file:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml">wp:
image: wordpress
ports:
- 8081:80
environment:
WORDPRESS_DB_HOST: wp1.database.prod
WORDPRESS_DB_USER: root
WORDPRESS_DB_PASSWORD: help_me
links:
- wp1.database.prod:mysql
mysql:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: help_me</code></pre></figure>
<p>Now we can run</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">docker-compose build .
docker-compose up</code></pre></figure>
<p>To prepare and start our infrastructure. Now we have one wordpress with own
mysql that run on port 8081. We can change wordpress port to start new isolate
wordpress installation.</p>
<p class="text-center">
<iframe src="//giphy.com/embed/l41lYCDgxP6OFBruE" width="480" height="268" frameborder="0" class="giphy-embed" allowfullscreen=""></iframe><p><a href="https://giphy.com/gifs/foxtv-win-ricky-gervais-emmys-2015-l41lYCDgxP6OFBruE">via
GIPHY</a></p>
</p>
<h2 id="in-cloud-with-aws-ecs">In Cloud with AWS ECS</h2>
<p>We won a battle but the war is too long, we can not use our PC as server. In
this article I propose <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html">AWS Elastic Container
Service</a>
a new AWS service that helps us to manage containers, why this service? Because
it is Docker and Docker Composer like, it’s managed by AWS, maybe there are
more flexible solutions, Swarm, Kubernetes but it is a good start point.</p>
<p><img src="/img/posts/2015-12-16/ecs.png" alt="AWS Elastic Container Service" /></p>
<p>A services of keywords to understand how it works:</p>
<ul>
<li><strong>Container instance</strong>: An Amazon EC2 that is running the Amazon ECS Agent. It has been registered into the ECS.</li>
<li><strong>Cluster</strong>: It is a pool of Container instances</li>
<li><strong>Task definition</strong>: A description of an application that contains one or more container definitions</li>
<li>Each Task definition running is a <strong>Task</strong></li>
</ul>
<h3 id="in-practice">In practice</h3>
<ol>
<li>Create a cluster</li>
</ol>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">ecs-cli configure \
--region eu-west-1 \
--cluster wps \
--access-key apikey \
--secret-key secreyKey</code></pre></figure>
<ol>
<li>Up nodes (one in this case)</li>
</ol>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">ecs-cli up --keypair key-ecs \
--capability-iam \
--size 1 \
--instance-type t2.medium</code></pre></figure>
<ol>
<li>Push your first task!</li>
</ol>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">ecs-cli compose --file docker-compose.yml \
--project-name wp1 up</code></pre></figure>
<ol>
<li>Follow the status of your tasks</li>
</ol>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">ecs-cli ps</code></pre></figure>
<p>You can use another docker-compose.yml with a different wordpress port to build
another task with another worpdress!</p>
<h2 id="now-is-only-a-problem-of-url">Now is only a problem of URL</h2>
<p>We are different isolated worpdress online, but they are an ip and different
ports, maybe our customers would use a domain name for example.
I don’t know if this solution is ready to run in production and it is good to
run more and more wordpress but a good service to turn and proxy requests is
HaProxy. This is an example of configuration for our use case:</p>
<p>wp1.gianarb.it and wp1.gianarb.it are two our customers and 54.229.190.73:8080,
54.229.190.73:8081 are our wordpress.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">...
frontend wp_mananger
bind :80
acl host_wp1 hdr(host) -i wp1.gianarb.it
acl host_wp2 hdr(host) -i wp2.gianarb.it
use_backend backend_wp1 if host_wp1
use_backend backend_wp2 if host_wp2
backend backend_wp1
server server1 54.229.190.73:8080 check
backend backend_wp2
server server2 54.229.190.73:8081 check</code></pre></figure>
<p>Note: This configuration increase the scalability of our system, because we can
add other service in order to support more traffic.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">backend backend_wp1
server server1 54.229.190.73:8080 check
server server1 54.229.190.12:8085 check
server server1 54.229.190.15:80 check</code></pre></figure>
<h3 id="there-are-other-solutions">There are other solutions</h3>
<ul>
<li>Nginx</li>
<li>Consul to increase the stability and the scalability of our endpoint</li>
</ul>
<div class="alert alert-info" role="alert">
This article is based on my presentation at <a href="https://gianarb.it/codemotion-2015/" target="_blank">Codemotion 2015</a>
</div>
<div class="alert alert-success" role="alert">
Thanks for review <a href="https://twitter.com/fntlnz" target="_blank">Lorenzo</a>! I'm in Ireland from 3 weeks but I am not ready to
write an article without your english review!
</div>
FastEventManager, only an event managerFastEventManager is a PHP library designed to be a smart and light event manager. You can use it in your applications or as a base component for your framework. It adds capabilities around events as attach and triggering of events.https://gianarb.it/img/github.png2015-11-01T00:00:00+00:002015-11-01T00:00:00+00:00https://gianarb.it/blog/fast-event-manager-only-an-event-manager<blockquote>
<p>The Event-Driven Messaging is a design pattern, applied within the
service-orientation design paradigm in order to enable the service consumers,
which are interested in events that occur within the periphery of a service
provider, to get notifications about these events as and when they occur
without resorting to the traditional inefficientpolling based mechanism.
by. <a href="https://en.wikipedia.org/wiki/Event-Driven_Messaging">wiki</a></p>
</blockquote>
<p>In PHP there are different implementation of this pattern, but <a href="https://github.com/gianarb/fast-event-manager">I tried to write
my idea</a>.
An easy to understand and to extends event manager based on regex.</p>
<p>Why? Because it is a good way to match strings, it is flexible and powerful.
As it is smart and little and it can be used as basis for custom implementation.
it resolves a regex and triggers events It supports a priority to order
triggered listeners.</p>
<h2 id="install">Install</h2>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">composer require gianarb/fast-event-manager</code></pre></figure>
<h2 id="getting-started">Getting Started</h2>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
require __DIR__."/vendor/autoload.php";
use GianArb\FastEventManager;
$eventManager = new FastEventManager();
$eventManager->attach("user_saved", function($event) {
});
$user = new Entity\User();
$eventManager->trigger("/user_saved/", $event);</code></pre></figure>
<p>Each listener has a priority (default = 0), it describe the order of execution</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
$eventManager->attach("wellcome", function() {
echo " dev!";
}, 100);
$eventManager->attach("wellcome", function() {
echo "Hello";
}, 345);
$eventManager->trigger("/wellcome/");
//output "Hello dev!"</code></pre></figure>
<p>I wrote this library because there are a lot of solutions that implement this
pattern but they are verbose, this is only an event manager if you search other
features you can extends it or you can use differents implementations.
On top of this library you can write your library to build an event manager ready
to use with your team in your applications.</p>
<p>This is a good solution because it is easy, ~31 line of code to trigger events
without fear to inherit many line of codes and unused features to maintain.</p>
Penny PHP framework made of componentsPenny a PHP framework made of components, write your microframework made of symfony, zend framework and other components.https://gianarb.it/img/penny.jpg2015-10-27T23:08:27+00:002015-10-27T23:08:27+00:00https://gianarb.it/blog/penny-framework-made-of-components<blockquote class="twitter-tweet tw-align-center" lang="en"><p lang="en" dir="ltr"><a href="https://twitter.com/hashtag/pennyphp?src=hash">#pennyphp</a> <a href="https://t.co/tsA2nE09GM">https://t.co/tsA2nE09GM</a> Why and what?! o.O <a href="https://twitter.com/hashtag/php?src=hash">#php</a> <a href="https://twitter.com/hashtag/framework?src=hash">#framework</a> to build <a href="https://twitter.com/hashtag/microservices?src=hash">#microservices</a> and application "consciously"</p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/659762064446083073">October 29, 2015</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p class="text-center">
<iframe src="https://ghbtns.com/github-btn.html?user=pennyphp&repo=penny&type=star&count=true&size=large" frameborder="0" scrolling="0" width="160px" height="30px"></iframe>
</p>
<p>The PHP ecosystem is mature, there are a lot of libraries that help you to write
good and custom applications. Too much libraries require a strong knowledge to
avoid the problem of maintainability and they also open a world made on specific
implementations for specific use cases.</p>
<p>A big framework adds a big overhead under your business logic sometimes, and some
of those unused features could cause maintainability problems and chaos.</p>
<p>Spending too much time reading the docs could be a problem, do you think you are
a system integrator and not a developer?! These are different works!</p>
<p>We are writing <a href="https://github.com/pennyphp/penny">penny</a> to share this idea.
This is a middleware, event driven framework to build the perfect
implementation for your specific project. The starting point we chose is made of:</p>
<ul>
<li><a href="https://github.com/zendframework/zend-diactoros">Zend\Diactoros</a> PSR-7 HTTP
library</li>
<li><a href="https://github.com/zendframework/zend-eventmanager">Zend\EventManager</a> to
design the application flow</li>
<li><a href="https://php-di">PHP-DI</a> DiC library</li>
<li><a href="https://github.com/nikic/FastRoute">FastRouter</a> because it is fast and easy to
use</li>
</ul>
<p>but we are working to replace every part of penny with the libraries perfect
for your use case.</p>
<p>Are you curious to try this idea? We are writing a big documentation around penny.
<a href="https://docs.pennyphp.org/en/latest/">docs.pennyphp.org/en/latest</a></p>
<p>And we have a set of use cases:</p>
<ul>
<li><a href="https://github.com/pennyphp/penny-classic-app">pennyphp/penny-classic-app</a>
builds with plates</li>
<li><a href="https://github.com/pennyphp/bookshelf">pennyphp/bookshelf</a> builds with
doctrine, twig</li>
<li><a href="https://github.com/gianarb/twitter-uservice">gianarb/twitter-uservice</a> gets
the last tweet from <code>#AngularConf15</code> hashtag</li>
</ul>
<p><a href="https://github.com/pennyphp/penny/issues?utf8=%E2%9C%93&q=is%3Aissue">Share your experience!</a></p>
vim composer 0.3.0 is readyvim-composer is a plugin to manage integration between composer and vimhttps://gianarb.it/img/vim.png2015-09-15T23:08:27+00:002015-09-15T23:08:27+00:00https://gianarb.it/blog/php-and-vim-composer-release-0-3-0<blockquote align="center" class="twitter-tweet" data-cards="hidden" lang="en"><p lang="en" dir="ltr"><a href="https://twitter.com/hashtag/vimForPHP?src=hash">#vimForPHP</a> <a href="https://t.co/EdczdpCrRc">https://t.co/EdczdpCrRc</a> <a href="https://twitter.com/hashtag/php?src=hash">#php</a> <a href="https://twitter.com/hashtag/vim?src=hash">#vim</a> Release 0.3.0 <a href="https://twitter.com/hashtag/composer?src=hash">#composer</a> plugin is ready! Thanks <a href="https://twitter.com/sensorario">@sensorario</a> for your work!</p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/641674841192574976">September 9, 2015</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>I’m very happy to announce release 0.3.0 of <a href="https://github.com/vim-php/vim-composer">vim-composer</a>.
This plugin builds a good integration between VIM and <a href="https://getcomposer.org">composer</a> the strong dependency manager for PHP.</p>
<h2 id="changelog">Changelog</h2>
<ul>
<li><a href="https://github.com/vim-php/vim-composer/pull/18">#18</a> Added missing ComposerUpdate function</li>
<li><a href="https://github.com/vim-php/vim-composer/pull/21">#21</a> Added missing CONTRIBUTING.md file</li>
<li><a href="https://github.com/vim-php/vim-composer/pull/20">#20</a> Require and/or init commands</li>
</ul>
<p>Now this plugin serve new function to require specific package. Update it and map new function <code>:ComposerRequireFunc</code>.</p>
Staging environment on demand with AWS CloudformationEnvironment are ephemeral. They come and go really quickly based on needs. AWS delivers a service called CloudFormation that allows you to easy describe via JSON or YALM specification a lot of AWS resources like EC2, Route53 hosted zone and domains, RDS, VPC, subnet and almost everything you normally do via console. This is infrastructure as code applied to AWS resource allows you to version and push on git entire AWS environment. You can replicate it over and over.https://gianarb.it/img/amazon-aws-logo.jpg2015-07-08T09:08:27+00:002015-07-08T09:08:27+00:00https://gianarb.it/blog/stagin-environment-on-demand-with-aws-cloudformation<blockquote class="twitter-tweet tw-align-center" lang="en"><p lang="en" dir="ltr">Staging environment on demand. To Work on <a href="https://twitter.com/hashtag/AWS?src=hash">#AWS</a> low level with <a href="https://twitter.com/hashtag/cloudformation?src=hash">#cloudformation</a> <a href="https://t.co/VWBR129637">https://t.co/VWBR129637</a> <a href="https://twitter.com/hashtag/cloud?src=hash">#cloud</a> <a href="https://twitter.com/hashtag/devops?src=hash">#devops</a></p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/621691855810494464">July 16, 2015</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<h2 id="staging-environment">Staging Environment</h2>
<p>There are few environments during my developer workflow, today I chose a little example:</p>
<ul>
<li>Production enviroment always exists, it runs the stable application and you can not use it for your test.</li>
<li><strong>Staging</strong> enviroment is a “pre-production” state.</li>
<li>Develop enviroment is instable and it runs new features and fixes, here there’s the work of all team but it’s not ready to go in production.</li>
</ul>
<p><img src="/img/cloudformation-staging/staging.jpg" alt="Staging graph" /></p>
<p>Staging environment in my opinion could be “volatile” version, we use it when our product is ready to go in production for the last time it was unused. Maybe this statement isn’t real in your work but if you think a little team of consultants that
work on different projects maybe this words have a sense.</p>
<h2 id="aws-cloudformation">AWS Cloudformation</h2>
<p>CloudFormation is an AWS service that helps you to orchestate all AWS services, you can write a template in JSON and you can use it to create an infrastructure with one click.
This solution helps me to build and destroy this environment and we can pay it only if it’s necessary, if you use a <code>stagin env == production env</code> it can be very very expensive.
This solution could help you to down cost.</p>
<h2 id="current-infrastructure">Current infrastructure</h2>
<p><img src="/img/cloudformation-staging/infra.jpg" alt="RDS and EC2 infrastructure" /></p>
<p>This is my template to build a simple application Frontend + MySQL (RDS).
In this implementation I build network configuration and I create one instance of RDS and one EC2 (my frontend).
<code>Parameters</code> key is the list of external parameters that I can use to configure my template, for example database and EC2 key pair, my root’s password..
<code>Resources</code> key contains description of all actors of this infrastructure.</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">{
"Parameters" : {
"VPCName" : {
"Type" : "String",
"Default" : "staging",
"Description" : "VPC name"
},
"ProjectName" : {
"Type" : "String",
"Default" : "app",
"Description" : "Project name"
},
"WebKey" : {
"Type" : "String",
"Default" : "web-key",
"Description" : "Ssh key to log into the web instances"
},
"WebInstanceType" : {
"Type" : "String",
"Default" : "m3.medium",
"Description" : "Web instance type"
},
"WebInstanceImage" : {
"Type" : "String",
"Default" : "ami-47a23a30",
"Description" : "Web instance image"
},
"DatabaseInstanceType" : {
"Type" : "String",
"Default" : "db.m3.medium",
"Description" : "Database instance type"
},
"DatabaseName" : {
"Type" : "String",
"Default" : "mydb",
"Description" : "Database instance's name"
},
"DatabaseMasterUsername" : {
"Type" : "String",
"Default" : "gianarb",
"Description" : "Name of master user"
},
"DatabaseEngineVersion" : {
"Type" : "String",
"Default" : "5.6",
"Description" : "MySQL version"
},
"DatabaseUserPassword" : {
"Type" : "String",
"Default" : "test1234",
"Description" : "User password"
},
"DatabasePublicAccess" : {
"Type" : "String",
"Default" : true
},
"DatabaseMultiAZ" : {
"Type" : "String",
"Default" : false
}
},
"Resources" : {
"Staging": {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.15.0.0/16",
"EnableDnsSupport" : true,
"EnableDnsHostnames" : true,
"InstanceTenancy" : "default",
"Tags" : [{"Key": "Name", "Value": {"Ref": "VPCName"}}]
}
},
"DatabaseSubnet1": {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"AvailabilityZone" : "eu-west-1a",
"CidrBlock" : "10.15.1.0/28",
"MapPublicIpOnLaunch" : true,
"VpcId": {
"Ref" : "Staging"
},
"Tags": [{"Key": "Name", "Value": "db-1a"}]
}
},
"DatabaseSubnet2": {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"AvailabilityZone" : "eu-west-1b",
"CidrBlock" : "10.15.1.16/28",
"MapPublicIpOnLaunch" : true,
"VpcId": {
"Ref" : "Staging"
},
"Tags" : [{"Key": "Name", "Value": "db-1b"}]
}
},
"WebSubnet1": {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"AvailabilityZone" : "eu-west-1a",
"CidrBlock" : "10.15.0.8/28",
"MapPublicIpOnLaunch" : true,
"VpcId": {
"Ref" : "Staging"
},
"Tags" : [{"Key": "Name", "Value": "web-1a"}]
}
},
"RDSSubnet": {
"Type" : "AWS::RDS::DBSubnetGroup",
"Properties" : {
"DBSubnetGroupDescription": "db-prod-subnet-group",
"SubnetIds" : [
{ "Ref": "DatabaseSubnet1" },
{ "Ref": "DatabaseSubnet2" }
]
}
},
"Database": {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage": "5",
"AllowMajorVersionUpgrade" : false,
"DBInstanceClass": {"Ref":"DatabaseInstanceType"},
"DBName" : {"Ref":"DatabaseName"},
"DBInstanceIdentifier": {"Ref":"DatabaseName"},
"Engine" : "MySQL",
"EngineVersion" : {"Ref":"DatabaseEngineVersion"},
"DBSubnetGroupName": {
"Ref": "RDSSubnet"
},
"MasterUsername" : {"Ref": "DatabaseMasterUsername"},
"MasterUserPassword" : {"Ref": "DatabaseUserPassword"},
"MultiAZ" : true,
"VPCSecurityGroups": [
{
"Ref": "DatabaseSG"
}
],
"PubliclyAccessible" : {"Ref": "DatabasePublicAccess"},
"Tags" : [{"Key": "Name", "Value": {"Fn::Join":[".", ["db", {"Ref": "ProjectName"}, {"Ref":"VPCName"}]]} }]
}
},
"WebInstance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : {"Ref": "WebInstanceImage"},
"InstanceType" : {"Ref": "WebInstanceType"},
"KeyName" : {"Ref": "WebKey"},
"BlockDeviceMappings" : [
{
"DeviceName" : "/dev/sdm",
"Ebs" : {
"VolumeType" : "io1",
"Iops" : "200",
"DeleteOnTermination" : "false",
"VolumeSize" : "20"
}
},
{
"DeviceName" : "/dev/sdk",
"NoDevice" : {}
}
],
"SubnetId": { "Ref" : "WebSubnet1" },
"SecurityGroupIds": [
{"Ref": "WebSG"}
]
}
},
"StagingZone": {
"Type" : "AWS::Route53::HostedZone",
"Properties" : {
"Name" : {"Fn::Join":[".", [{"Ref": "ProjectName"}, {"Ref":"VPCName"}]]},
"VPCs" : [{"VPCId": {"Ref": "Staging"}, "VPCRegion": "eu-west-1"}]
}
},
"StagingInternetGateway" : {
"Type" : "AWS::EC2::InternetGateway",
"Properties" : {
"Tags" : [ {"Key" : "Name", "Value" : {"Fn::Join":["-", [{"Ref":"VPCName"}, "igw"]]}}]
}
},
"StagingIgwAttach": {
"Type" : "AWS::EC2::VPCGatewayAttachment",
"Properties" : {
"InternetGatewayId" : {"Ref": "StagingInternetGateway"},
"VpcId" : {"Ref": "Staging"}
}
},
"StagingRouteTable": {
"Type" : "AWS::EC2::RouteTable",
"Properties" : {
"VpcId" : {"Ref": "Staging"}
}
},
"LocalRoute": {
"Type" : "AWS::EC2::Route",
"Properties" : {
"DestinationCidrBlock" : "0.0.0.0/0",
"GatewayId" : {"Ref": "StagingInternetGateway"},
"RouteTableId" : {"Ref": "StagingRouteTable"}
}
},
"Web1LocalRoute": {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"RouteTableId" : {"Ref": "StagingRouteTable"},
"SubnetId" : {"Ref": "WebSubnet1"}
}
},
"Db1LocalRoute": {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"RouteTableId" : {"Ref": "StagingRouteTable"},
"SubnetId" : {"Ref": "DatabaseSubnet1"}
}
},
"Db2LocalRoute": {
"Type" : "AWS::EC2::SubnetRouteTableAssociation",
"Properties" : {
"RouteTableId" : {"Ref": "StagingRouteTable"},
"SubnetId" : {"Ref": "DatabaseSubnet2"}
}
},
"DatabaseSG": {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Database security groups",
"SecurityGroupIngress" : [
{
"IpProtocol" : "tcp",
"FromPort": 3306,
"ToPort" : "3306",
"SourceSecurityGroupId": {"Ref" : "WebSG"}
}
],
"Tags" : [{"Key": "Name", "Value": "db-sg"}],
"VpcId" : {"Ref": "Staging"}
}
},
"WebSG": {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Web security groups",
"SecurityGroupIngress" : [
{
"IpProtocol" : "tcp",
"ToPort" : 80,
"FromPort": 80,
"CidrIp" : "0.0.0.0/0"
},
{
"IpProtocol" : "tcp",
"ToPort" : 22,
"FromPort": 22,
"CidrIp" : "0.0.0.0/0"
}
],
"Tags" : [{"Key": "Name", "Value": "web-sg"}],
"VpcId" : {"Ref": "Staging"}
}
},
"DatabaseRecordSet" : {
"Type" : "AWS::Route53::RecordSet",
"Properties" : {
"HostedZoneId" : {
"Ref": "StagingZone"
},
"Comment" : "DNS name for database",
"Name" : {"Fn::Join":[".", ["db", {"Ref": "ProjectName"}, {"Ref":"VPCName"}]]},
"Type" : "CNAME",
"TTL" : "300",
"ResourceRecords" : [
{ "Fn::GetAtt" : [ "Database", "Endpoint.Address"]}
]
}
}
}
}</code></pre></figure>
<h2 id="conclusion">Conclusion</h2>
<p>You can load this teamplate in your account and after environment creations you are ready to work with one EC2 instance and one RDS with MySQL 5.6 installed.
You can log into the web interface with key-pair chosen during the creation flow (default ga-eu) and I set default this mysql credential:</p>
<ul>
<li>user gianarb</li>
<li>password test1234</li>
</ul>
<p>But you can change it before running this template because they are <code>Parameters</code>.
This approach in my opinion is very powerful because you can start versioning your infrastructure and you can delete and restore it quickly because if you delete the cloudformation stack it rollbacks all resources, it is very easy!</p>
<h2 id="trick">Trick</h2>
<p>Parameters node create a form into the AWS CloudFormation console to choose a lot of different variable values, for example name of intances or key-pair to log in your EC2.</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">{
"Parameters" : {
"VPCName" : {
"Type" : "String",
"Default" : "staging",
"Description" : "VPC name"
},
"ProjectName" : {
"Type" : "String",
"Default" : "app",
"Description" : "Project name"
},
"WebKey" : {
"Type" : "String",
"Default" : "web-key",
"Description" : "Ssh key to log into the web instances"
}
}</code></pre></figure>
<hr class="style-two" />
<p>Resources node contains all elements of your infrastructure, EC2, RDS, VCP.. You can use the parameteters with a simple <code>Ref Key</code>.
es. <code>[{"Key": "Name", "Value": "ProjectName"}]</code> describe the name of the specific project into the parameter form.</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">{
"Resources" : {
"Staging": {
"Type" : "AWS::EC2::VPC",
"Properties" : {
"CidrBlock" : "10.15.0.0/16",
"EnableDnsSupport" : true,
"EnableDnsHostnames" : true,
"InstanceTenancy" : "default",
"Tags" : [{"Key": "Name", "Value": {"Ref": "VPCName"}}]
}
},
"DatabaseSubnet1": {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"AvailabilityZone" : "eu-west-1a",
"CidrBlock" : "10.15.1.0/28",
"MapPublicIpOnLaunch" : true,
"VpcId": {
"Ref" : "Staging"
},
"Tags": [{"Key": "Name", "Value": "db-1a"}]
}
}
}</code></pre></figure>
<hr class="style-two" />
<p>In your template you can describe VPC and create its subnet. You can also describe the specific resource and you can use it to build another</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">WebSubnet1": {
"Type" : "AWS::EC2::Subnet",
"Properties" : {
"AvailabilityZone" : "eu-west-1a",
"CidrBlock" : "10.15.0.8/28",
"MapPublicIpOnLaunch" : true,
"VpcId": {
"Ref" : "Staging"
},
"Tags" : [{"Key": "Name", "Value": "web-1a"}]
}
},</code></pre></figure>
<p>In this example I resumed <code>Staging</code> VPC to build its subnet.</p>
<hr class="style-two" />
<p>This chapter is insteresting because it creates a RecordSet to map a CNAME DNS in your VPC and now in your Web instances you can resolve MYSql host with <code>db.app.staging</code>.</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">"DatabaseRecordSet" : {
"Type" : "AWS::Route53::RecordSet",
"Properties" : {
"HostedZoneId" : {
"Ref": "StagingZone"
},
"Comment" : "DNS name for database",
"Name" : {"Fn::Join":[".", ["db", {"Ref": "ProjectName"}, {"Ref":"VPCName"}]]},
"Type" : "CNAME",
"TTL" : "300",
"ResourceRecords" : [
{ "Fn::GetAtt" : [ "Database", "Endpoint.Address"]}
]
}
}</code></pre></figure>
<p><br />
<br />
<br /></p>
<div class="well"><a target="_blank" href="https://twitter.com/EmanueleMinotto">@EmanualeMinotto</a> thanks for trying to fix my bad English</div>
Build your Zend Framework Console ApplicationZF Console is a component written by zf-campus and Apigility organization that help you to build console application using different Zend Framework componentshttps://gianarb.it/img/zf.jpg2015-05-21T23:08:27+00:002015-05-21T23:08:27+00:00https://gianarb.it/blog/zendframework-console-app<blockquote class="twitter-tweet tw-align-center" lang="en"><p lang="en" dir="ltr">Blogpost about console-skeleton-app for your console application <a href="https://t.co/WuVq0GZlxE">https://t.co/WuVq0GZlxE</a> <a href="https://twitter.com/hashtag/PHP?src=hash">#PHP</a> <a href="https://twitter.com/hashtag/ZF?src=hash">#ZF</a> <a href="https://twitter.com/hashtag/console?src=hash">#console</a> <a href="https://twitter.com/hashtag/develop?src=hash">#develop</a></p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/613292048708468736">June 23, 2015</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<div class="alert alert-success" role="alert"><strong>Github: </strong>Article written about <a target="_blank" href="https://github.com/gianarb/console-skeleton-app">console-skeleton-app</a> 1.0.0</div>
<p>I’m writing a skeleton app to build console/bash application in PHP.
This project is very easy and it depends on ZF\Console a zfcampus project and Zend\Console builds by ZF community.
I have a todo list for the future but for the time being it’s just a blog post about these two modules.</p>
<ul>
<li>Integration with container system to manage dependency injection</li>
<li>Docs to test your command</li>
<li>Use cases and different implementations</li>
</ul>
<h2 id="zfconsole-and-other-components">ZF\Console and other components</h2>
<ul>
<li><a href="https://github.com/zfcampus/zf-console">ZF\Console</a> is maintained by zfcampus and it is used by Apigility</li>
<li><a href="https://github.com/zendframework/zend-console">zendframework\zend-console</a> is maintained by zendframework, all the info are in the <a href="https://framework.zend.com/manual/current/en/modules/zend.console.introduction.html">documantation</a></li>
</ul>
<h2 id="tree">Tree</h2>
<p>This is my folders structure proposal, there are three entrypoint in the <code>bin</code> directory, one for bash, one for php and a bat for Window.
I use composer to manage my dependencies and I included .lock file because this project is an APPLICATION not a library..
<code>/config</code> directory contains only routing definitions but in the future we can add services and other configurations.
<code>src/Command/</code> contains my commands.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">├── bin
│ └── console.php
├── composer.json
├── composer.lock
├── config
│ └── routes.php
├── src
│ └── Command
│ ├── Conf.php
│ ├── Database.php
│ └── Download.php
└── vendor
└── ...</code></pre></figure>
<h2 id="bootstrap">Bootstrap</h2>
<p>The Application’s entrypoints are just example and they require few changes.
First we have to change the version in the parameters.php configuration file and also change the application name <code>'app'</code> to what fits.
To load configurations from different sources I will use the well known <code>Zend\Config</code> component.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
require __DIR__.'/../vendor/autoload.php';
use Zend\Console\Console;
use ZF\Console\Application;
use ZF\Console\Dispatcher;
$version = '0.0.1';
$application = new Application(
'app',
$version,
include __DIR__ . '/../config/routes.php',
Console::getInstance(),
new Dispatcher()
);
$exit = $application->run();
exit($exit);</code></pre></figure>
<h2 id="routes">Routes</h2>
<p><code>config/routes.php</code> contains router configurations. This is just an example but you can see all options <a href="https://github.com/zfcampus/zf-console#defining-console-routes">here</a>.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
return [
[
'name' => 'hello',
'route' => "--name=",
'short_description' => "Good morning!! This is a beautiful day",
"handler" => ['App\Command\Hello', 'run'],
],
];</code></pre></figure>
<h2 id="command">Command</h2>
<p>Basic command to wish you a good day!
I decided that a command doesn’t extends any class because in my opinion is a good way to impart readability and simplicity.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
namespace App\Command;
use ZF\Console\Route;
use Zend\Console\Adapter\AdapterInterface;
class Hello
{
public static function run(Route $route, AdapterInterface $console)
{
$name = $route->getMatchedParam("name", "@gianarb");
$console->writeLine("Hi {$name}, you have call me. Now this is an awesome day!");
}
}</code></pre></figure>
<h2 id="troubleshooting-and-tricks">Troubleshooting and tricks</h2>
<ul>
<li>OSx return an error because zf-console use a function blocked into the mac os php installation. Have a look at PR<a href="https://github.com/zfcampus/zf-console/pull/22">#22</a></li>
<li>See <a href="https://www.sitepoint.com/packaging-your-apps-with-phar/">this</a> article to package your application in a phar archive..</li>
</ul>
<p><br />
<br />
<br /></p>
<div class="well"><a target="_blank" href="https://twitter.com/__debo">@__debo</a> thanks for trying to fix my bad English</div>
Test your Symfony Controller and your service with PhpUnitTest your Symfony Controller with PhpUnit. You expect that if one parameter is true your action get a service by Dependence Injcation and use it!https://gianarb.it/img/symfony.png2015-05-21T23:08:27+00:002015-05-21T23:08:27+00:00https://gianarb.it/blog/symfony-unit-test-controller-with-phpunit<blockquote align="center" class="twitter-tweet" lang="en"><p lang="en" dir="ltr">Unit <a href="https://twitter.com/hashtag/test?src=hash">#test</a> for your <a href="https://twitter.com/hashtag/Controller?src=hash">#Controller</a> with <a href="https://twitter.com/hashtag/PhpUnit?src=hash">#PhpUnit</a> and <a href="https://twitter.com/hashtag/Symfony?src=hash">#Symfony</a>.. With a little use case of <a href="https://twitter.com/hashtag/DepedenceInjaction?src=hash">#DepedenceInjaction</a> test <a href="https://t.co/JNb39EyRly">https://t.co/JNb39EyRly</a> <a href="https://twitter.com/hashtag/php?src=hash">#php</a></p>— Gianluca Arbezzano (@GianArb) <a href="https://twitter.com/GianArb/status/601526550438215680">May 21, 2015</a></blockquote>
<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>In this article I would like share with you a little experience with:</p>
<ul>
<li>Symfony MVC</li>
<li>PhpUnit</li>
<li>Symfony Dependence Injaction</li>
</ul>
<p>This is an example of very easy controller.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
namespace AppBundle\Controller;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\Component\HttpFoundation\Request;
class SomeStuffController extends FOSRestController
{
/**
* @Rest\Post("/go")
* @return array
*/
public function goAction(Request $request)
{
if($this->container->getParameter("do_stuff")) {
$body = $this->container->get("stuff.service")->splash($request->getContent());
}
return [];
}
}</code></pre></figure>
<p><code>$this->container->getParameter("do_stuff")</code> is a boolean parameter that enable or disable a feature, How can I test this snippet?
I can try to write a functional test but in my opinion is easier write a series of unit tests with PhpUnit to validate my expectations.</p>
<h2 id="expectations">Expectations</h2>
<ul>
<li>If <code>do_stuff</code> parameter is false function get by my container will be call zero times</li>
<li>If <code>do_stuff</code> parameter is true function get by my container will be call one times</li>
</ul>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
namespace AppBundle\Tests\Controller;
use Liip\FunctionalTestBundle\Test\WebTestCase;
use AppBundle\Controller\SomeStuffController;
class SomeStuffControllerTest extends WebTestCase
{
public function testDoStuffIsTrue()
{
$request = $this->getMock("Symfony\Component\HttpFoundation\Request");
$container = $this->getMock("Symfony\Component\DependencyInjection\ContainerInterface");
$service = $this->getMockBuilder("Some\Stuff")->disableOriginalConstructor()->getMock();
$container->expects($this->once())
->method("getParameter")
->with($this->equalTo('do_stuff'))
->will($this->returnValue(true));
$container->expects($this->once())
->method("get")
->with($this->equalTo('stuff.service'))
->will($this->returnValue($service));
$controller = new SameStuffController();
$controller->setContainer($container);
$controller->goAction($request);
}
}</code></pre></figure>
<p>This is my first expetection “If <code>do_stuff</code> param is true I call <code>stuff.service</code>”.
In this controller I use a few objects, Http\Request, Container and <code>stuff.service</code> in this example is a <code>Some\Stuff</code> class.
In the first step I have created one mock for each object.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
$request = $this->getMock("Symfony\Component\HttpFoundation\Request");
$container = $this->getMock("Symfony\Component\DependencyInjection\ContainerInterface");
$service = $this->getMockBuilder("Some\Stuff")->disableOriginalConstructor()->getMock();</code></pre></figure>
<p>In the second step I have written my first expetctation, “Call only one time function <code>getParameter</code> from <code>$container</code> with argument do_stuff and it returns true”.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
$container->expects($this->once())
->method("getParameter")
->with($this->equalTo('do_stuff'))
->will($this->returnValue(true));</code></pre></figure>
<p>Thanks at this definitions I know that there will be another effect, my action will call only one time <code>$container->get("stuff.service")</code> and it will be return an Some\Stuff object.</p>
<p>The second test that we can write is “if <code>do_stuff</code> is false <code>$contaner->get("stuff.service")</code> it will not be called.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
public function testDoStuffIsFalse()
{
$request = $this->getMock("Symfony\Component\HttpFoundation\Request");
$container = $this->getMock("Symfony\Component\DependencyInjection\ContainerInterface");
$service = $this->getMockBuilder("Some\Stuff")->disableOriginalConstructor()->getMock();
$container->expects($this->once())
->method("getParameter")
->with($this->equalTo('do_stuff'))
->will($this->returnValue(false));
$container->expects($this->never())
->method("get")
->with($this->equalTo('stuff.service'))
->will($this->returnValue($service));
$controller = new SameStuffController();
$controller->setContainer($container);
$controller->goAction($request);
}</code></pre></figure>
The price of modularityModularity is not only a beautiful word, It has roles and it is a methodology that helps you to keep your project easy to understand. Modularity is a key principle to have an easy on board of new developer in your application because it will look separated and simpler to approach. As a developer you should think about how your scaffolding and your code looks from the outside because in reality you read much more code compared with how code you write.https://gianarb.it/img/gianarb.png2015-02-21T23:08:27+00:002015-02-21T23:08:27+00:00https://gianarb.it/blog/the-price-of-modularity<p>Today all frameworks are <strong>modulable</strong> but it isn’t just a beautiful word, behind it there are a lot of concepts and ideas:</p>
<ul>
<li>The modularity helps you to reuse parts of code in different projects</li>
<li>Every component is indipendent so you work on single part of code</li>
<li>Every <strong>component</strong> solves a specific problem… it’s a beautiful concept that helps you with maintainance!</li>
<li>other stuffs..</li>
</ul>
<p>As you can imagine there is a drawback, all this requires a big effort.
Ideally every component requires personal circle of release, repository, commits, pull requests, travis conf, documentation etc. etc.</p>
<p>Anyway several shorcuts are available. For instance, <em>git subtree</em> could help you in this war but the key is this: you need an agreement to win.</p>
<p>Zend Framwork Community choose another street, <code>Zend\Mvc</code> in this moment required:</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json">{
"name": "zendframework/zend-mvc",
"...": "...",
"target-dir": "Zend/Mvc",
"require": {
"php": ">=5.3.23",
"zendframework/zend-eventmanager": "self.version",
"zendframework/zend-servicemanager": "self.version",
"zendframework/zend-form": "self.version",
"zendframework/zend-stdlib": "self.version"
},
"require-dev": {
"zendframework/zend-authentication": "self.version",
"zendframework/zend-console": "self.version",
"zendframework/zend-di": "self.version",
"zendframework/zend-filter": "self.version",
"zendframework/zend-http": "self.version",
"zendframework/zend-i18n": "self.version",
"zendframework/zend-inputfilter": "self.version",
"zendframework/zend-json": "self.version",
"zendframework/zend-log": "self.version",
"zendframework/zend-modulemanager": "self.version",
"zendframework/zend-session": "self.version",
"zendframework/zend-serializer": "self.version",
"zendframework/zend-text": "self.version",
"zendframework/zend-uri": "self.version",
"zendframework/zend-validator": "self.version",
"zendframework/zend-version": "self.version",
"zendframework/zend-view": "self.version"
},
"suggest": {
"zendframework/zend-authentication": "Zend\\Authentication component for Identity plugin",
"zendframework/zend-config": "Zend\\Config component",
"zendframework/zend-console": "Zend\\Console component",
"zendframework/zend-di": "Zend\\Di component",
"zendframework/zend-filter": "Zend\\Filter component",
"...": "..."
},
"...": "..."
}</code></pre></figure>
<p>A few <code>require-dev</code> dependencies are used into the component to run some features, why? This force me to think <em>“Dependencies of this feature are included or not?”</em>!!
Composer was born to solve it! In my opinion the cost of the question is highest than download a few unused classes.
There are a lot of unused classes? Maybe too much?</p>
<p>Even if the right answer donsn’t exist I think thant some indicators may help you to understand when is the moment to split the component:</p>
<ul>
<li>List of dependencies</li>
<li>Complexity of component</li>
<li>Features</li>
<li>..</li>
</ul>
<p>No shortcuts.</p>
Zend Framework release 2.3.4Zend Framework release 2.3.4https://gianarb.it/img/zf.jpg2015-01-14T00:00:00+00:002015-01-14T00:00:00+00:00https://gianarb.it/blog/zf2-release-234<p>Zend Framework 2.3.4 is ready! After 4 mouths the new path version of ZF2 is
published.</p>
<p>How all path release there aren’t new important feature but the list of <a href="https://github.com/zendframework/zf2/pulls?q=is%3Aclosed+is%3Apr+milestone%3A2.3.4+">pull
requests</a>
is very long.</p>
<ul>
<li><a href="https://github.com/zendframework/zf2/pull/7112">#7112</a> You can find into /resources directory a ZF official logo</li>
<li><a href="https://github.com/zendframework/zf2/pull/7087">#7087</a> Happy new year by ZF!</li>
<li><a href="https://github.com/zendframework/zf2/issues/6673">#6673</a> <code>Zend\Http\Header</code> now support DateTime format for expire Cookie</li>
</ul>
<p>Zend Framework follow <a href="https://semver.org/">semver</a> directives, <code>2.3.4</code> is a path
release in this version there are a log list of <a href="https://github.com/zendframework/zf2/pulls?q=is%3Aclosed+is%3Apr+milestone%3A2.3.4+label%3Abug">bug
fixes</a></p>
<p>Good download <a href="https://github.com/zendframework/zf2/releases/tag/release-2.3.4">Zend Femework
2.3.4</a>, this is
the
<a href="https://github.com/zendframework/zf2/blob/18534b6f2c14f52898bb208932fedacd5324be63/CHANGELOG.md">changelog</a></p>
Influx DB and PHP implementationInfluxDB is a popular and open source time series database capable to store millions of points keeping a fast lookup. It supports SQL as query language and it exposes an HTTP API to interact with it. In Corley we wrote a PHP SDK and we released it open source to integrate InfluxDB in your PHP application.https://gianarb.it/img/influxdb.png2015-01-06T00:00:00+00:002015-01-06T00:00:00+00:00https://gianarb.it/blog/InfluxDB-and-PHP<p>Influx DB is <a href="http://en.wikipedia.org/wiki/Time_series_database">time series
database</a> written in Go.</p>
<p>It supports SQL like queries and it has different entry points, REST API (tcp
protocol) and UDP.</p>
<div class="row">
<div class="col-md-4 col-md-offset-3"><img class="img-fluid" src="/img/influxdb.png" /></div>
</div>
<p>We wrote a <a href="https://github.com/corley/influxdb-php-sdk">sdk</a> to manage
integration between Influx and PHP.</p>
<p>It supports Guzzle Adapter but if you use Zend\Client you can write your
implementation.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
$guzzle = new \GuzzleHttp\Client();
$options = new Options();
$adapter = new GuzzleAdapter($guzzle, $options);
$client = new Client();
$client->setAdapter($adapter);</code></pre></figure>
<p>In this case we are using a Guzzle Client, we communicate with Influx in TPC, but we can speak with it in UDP</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
$options = new Options();
$adapter = new UdpAdapter($options);
$client = new Client();
$client->setAdapter($adapter);</code></pre></figure>
<p>Both of them have the same usage</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
$client->mark("app.search", $points, "s");</code></pre></figure>
<p>The first different between udp and tcp is known, TPC after request expects a
response, UDP does not expect anything and in this case does not exist any
delivery guarantee. If you can accept this stuff this is the benchmark:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">Corley\Benchmarks\Influx DB\AdapterEvent
Method Name Iterations Average Time Ops/second
------------------------ ------------ -------------- -------------
sendDataUsingHttpAdapter: [1,000 ] [0.0026700308323] [374.52751]
sendDataUsingUdpAdapter : [1,000 ] [0.0000436344147] [22,917.69026]</code></pre></figure>
Zf2 Event, base useIntegrate in your application an event system is a good way to decouple and extend your application keeping it clean and clear. An event manager allow you to trigger and catch event based on what your application do. You can for example send different kind of notifications like email, slack message and so on from the same event like 'user registration'. Zend Framework a popular and open source PHP framework has a component called EventManager that helps you do integrate such flow.https://gianarb.it/img/zf.jpg2013-11-21T12:38:27+00:002013-11-21T12:38:27+00:00https://gianarb.it/blog/Zf2-Event-base-use<p>Hi! Some mouths ago I have writted a gist for help me to remember a base use of
Events and Event Manager into Zend Fremework, in this article I report this
small tutorial.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
require_once __DIR__."/vendor/autoload.php";
class Foo
{
/* @var \Zend\EventManager\EventManagerInterface */
protected $eventManager;
public function getEventManager()
{
if(!$this->eventManager instanceof \Zend\EventManager\EventManagerInterface){
$this->eventManager = new \Zend\EventManager\EventManager();
}
return $this->eventManager;
}
public function echoHello()
{
$this->getEventManager()->trigger(__FUNCTION__."_pre", $this);
echo "Hello";
$this->getEventManager()->trigger(__FUNCTION__."_post", $this);
}
}
$foo = new Foo();
$foo->getEventManager()->attach('echoHello_pre', function($e){
echo "Wow! ";
});
$foo->getEventManager()->attach('echoHello_post', function($e){
echo ". This example is very good! \n";
});
$foo->getEventManager()->attach('echoHello_post', function($e){
echo "\nby gianarb92@gmail.com \n";
}, -10);
$foo->echoHello();</code></pre></figure>
<p>The result:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">gianarb@GianArb-2 eventTest :) $ php try.php
Wow! Hello. This example is very good!
by gianarb92@gmail.com</code></pre></figure>
<p><a href="https://framework.zend.com/manual/2.0/en/modules/zend.event-manager.event-manager.html">@see Zend Event Manager Ref</a></p>
Git global gitignoreGit is the most popular code version control. It helps you to manage and share your code, writing a history about the evolution of it over time. It also allow teams to work together managing conflicts and large codebases. Based on languages, projects or operating system there are files that you should never commit such as .DS_Store on mac. You can setup a user level gitignore file to keep them out.https://gianarb.it/img/git.png2013-11-21T12:38:27+00:002013-11-21T12:38:27+00:00https://gianarb.it/blog/Git-globa-gitignore<p><code>.gitignore </code> helps me manage my commits by setting which files or
directory don’t end in my repository. I know two good practices if you work for
example on an open source project:</p>
<ul>
<li>You don’t commit your IDE configurations</li>
<li>Not use .gitignore file for exclude IDE configuration, because this is
personal problem. There are differents IDE, if all devs exclude this files on
a repository level the lists is very long.</li>
</ul>
<p>I follow this practices for all my projects, if you are Mac user you have a
DS_STORE files, there is a method for exclude this file of default.</p>
<p><code>~./.gitconfig</code> is your configuration file, every user has it. If you execute this command</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">$. git config --global core.excludesfile ~/.gitignore_global</code></pre></figure>
<p>into this file it write thiese lines</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">[core]
excludesfile = /Users/gianarb/.gitignore_global</code></pre></figure>
<p><code>/Users/gianarb/.gitignore_global</code> is my global gitignore file!</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"># IDE #
#######
.idea
# COMPOSER #
############
composer.phar
# OS generated files #
######################
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db</code></pre></figure>
Vagrant Up, slide and first talkThis is my first public talk delivered at the PHP User Group Milan. It is about how to setup a local environment using Vagrant as automation tool. A well setup local environment is a must have to quick develop your application. With Vagrant you make infrastructure as a code to provision your environment. You can push your code inside a version control system such as GIT to share it with your collagues.https://gianarb.it/img/vagrant-logo.png2013-09-14T12:08:27+00:002013-09-14T12:08:27+00:00https://gianarb.it/blog/vagrant-up-talk-milano<h3>Vagrant Up</h3>
<p>Friday 12 Sept 2013 I talked at Vagrant, tool for manage VM, these is my slide.
I thanks <a href="https://milano.grusp.org/" target="_blank">PugMi</a> for this opportunity if you have questions I'm here! :grin: </p>
<iframe style="display:block; margin: 0 auto;" src="https://www.slideshare.net/slideshow/embed_code/26159972" width="597" height="486" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC;border-width:1px 1px 0;margin-bottom:5px" allowfullscreen webkitallowfullscreen mozallowfullscreen> </iframe> <div style="margin-bottom:5px;text-align:center;"> <strong> <a href="https://www.slideshare.net/GianlucaArbezzano/presentazione-def-26159972" title="Vagrant - PugMI" target="_blank">Vagrant - PugMI</a> </strong> from <strong><a href="https://www.slideshare.net/GianlucaArbezzano" target="_blank">Gianluca Arbezzano</a></strong> </div>
</br></br>
<blockquote class="twitter-tweet" align="center"><p>quasi 30 persone a sentire <a href="https://twitter.com/GianArb">
@GianArb</a> parlare di <a href="https://twitter.com/search?q=%23vagrant&src=hash">#vagrant</a> <a href="https://twitter.com/search?q=%23php&src=hash">#php</a> <a href="https://twitter.com/search?q=%23pugMi&src=hash">#pugMi</a> <a href="https://twitter.com/search?q=%23milano&src=hash">#milano</a> <a href="https://t.co/75MOJiZmDZ">pic.twitter.com/75MOJiZmDZ</a></p>— Milano PHP (@MilanoPHP) <a href="https://twitter.com/MilanoPHP/statuses/378213865072656385">September 12, 2013</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
Zend Framework 2 - Console usage a speed helpCLI tools are an easy way to interact with an application because you can drive users or even other developers in a well know direction. It is a very good way to decrease possible mistakes. Zend Framework 2 a PHP open source framework has a Console package that helps you to address common issue like argument management, command parsing and to format a colored and nice output.https://gianarb.it/img/zf.jpg2013-08-22T08:08:27+00:002013-08-22T08:08:27+00:00https://gianarb.it/blog/zf2-console-usage-speed-help<p>With Zend Framework is very easy to write a command line tool to manage
different things. But what if there are more commands? How do you remeber them
all?</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
namespace ModuleTest;
use Zend\Console\Adapter\AdapterInterface;
class Module {
public function getConsoleUsage(AdapterInterface $console)
{
return array(
array('test <params1> <params2> [--params=]', 'Description of test command'),
array('run <action>', 'Start anction')
);
}
}</code></pre></figure>
<p>You can write this function in a Module.php file, and create a basic helper to
help you see when you write a bad command.</p>
<p>English by Rali :smile: Thanks!!!! :smile:</p>
Generale Jekyll sitemap without pluginEvery site should have its sitemap to tell search engine like Google about news and updates from your site. With a static site generator such as Jekyll you need to generate statically the sitemap too. This article explains how to write a template that generate a sitemap.https://gianarb.it/img/jekyll.png2013-08-09T09:38:27+00:002013-08-09T09:38:27+00:00https://gianarb.it/blog/Generate-jekyll-sitemap-without-plugin<p>This blog is a static blog and uses GitHub pages, GitHub pages are generally
deployed using Jekyll.</p>
<h3 id="how-can-you-generate-a-sitemap-without-jekyll-plugin">How can you generate a sitemap without Jekyll plugin?</h3>
<p>This <a href="https://gist.github.com/GianArb/6172377">gist</a> answers your question.</p>
<p>I use some post values: changefreq, date and priority, if you don’t set any
specific values for them default values are used that are, 0.8 for priority and
month for frequency.</p>
<p>In a single post you add this params for use correct params!</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php">---
layout: post
title: "Why this blog?"
date: 2013-07-22 23:08:27
categories: me
tags: me, developer, presentation, gianarb
summary: Gianluca Arbezzano, developer, Italian, why open this blog?
changefreq: monthly
---</code></pre></figure>
<p>If you want to know more about the Sitemap Protocol read
<a href="https://www.sitemaps.org/protocol.html">this</a>.</p>
<p><a href="https://github.com/MarcoDeBortoli">Marco</a> thanks for English! :)</p>
Zend Framework 2 - How do you implement log service?Logging is a requirement for every application. In PHP and in every other language. It is the way your application has to tell you what its doing. This article is about how to implement a logger in a Zend Framework 2 application in PHP. This solution achieve simplicity and usability.https://gianarb.it/img/zf.jpg2013-07-26T23:08:27+00:002013-07-26T23:08:27+00:00https://gianarb.it/blog/how-do-you-implement-log-service<p>A log system is an essential element for any application. It is a way to check
the status and use of the application. For a basic implementation you can refer
to the fig-stanrdars organization
<a href="https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-3-logger-interface.md">PSR-3</a>
article, that describes th elogger interface.</p>
<p>Zend Framework 2 implement a <a href="https://github.com/zendframework/zf2/tree/master/library/Zend/Log">Logger
Component</a>,
the following is an example of how to use it with service manager.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
return array(
'service_manager' => array(
'abstract_factories' => array(
'Zend\Log\LoggerAbstractServiceFactory',
),
),
'log' => array(
'Log\App' => array(
'writers' => array(
array(
'name' => 'stream',
'priority' => 1000,
'options' => array(
'stream' => 'data/app.log',
),
),
),
),
),
);</code></pre></figure>
<p><a href="https://github.com/zendframework/zf2/blob/master/library/Zend/Log/LoggerServiceFactory.php">LoggerAbstractServiceFactory</a>
is a Service Factory, as an example, into service Manager class Logger and will
be used in the whole application. Log/App is the name of a single logger, and
writer is an adapter that is used to choose the method of writing, in this case
everything is written to file, but you can use a DB adapter and write your log
into database.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
namespace GianArb\Controller;
class GeneralController
extends AbastractActionController
{
public function testAction(){
$logger = $this->getServiceLocator()->get('Log\App');
$logger->log(\Zend\Log\Logger::INFO, "This is a little log!");
}
}</code></pre></figure>
<p>With this configuration Log\App writes a string into data/app.log file, with
INFO property. By default you can use an array of properties.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
protected $priorities = array(
self::EMERG => 'EMERG',
self::ALERT => 'ALERT',
self::CRIT => 'CRIT',
self::ERR => 'ERR',
self::WARN => 'WARN',
self::NOTICE => 'NOTICE',
self::INFO => 'INFO',
self::DEBUG => 'DEBUG',
);</code></pre></figure>
<p>Usage of different keys is a good practice because it is very easy to write
filter or log categories.</p>
<p>Another good practice, valid for all services in general, is to create your
class extending single service.</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><?php
use Zend\Log\Logger
class MyLogger extends Logger</code></pre></figure>
<p>This choice helps managing future customizations of services and is another
important layer for managing unexpected updates.</p>
<p>Rali, thanks for your help with my robotic english! :P</p>
Why this blog?This is my first article on Jekyll. A new static site generator that I have selected as engine for my site to replace wordpress.https://gianarb.it/img/myselfie.jpg-large2013-07-22T23:08:27+00:002013-07-22T23:08:27+00:00https://gianarb.it/blog/why-this-blog<p>Hi! I’m Gianluca aka <a href="https://twitter.com/gianarb">GianArb</a>, I’m web developer,
works with Php, SQL and noSql databases and in this time I’m crazy for DevOps,
Vagrant and Chef, than for manage this tool I’m learning Ruby.</p>
<h3 id="why-this-blog">Why this blog?</h3>
<p>I’m opening this blog because my English is very terrible! I have an Italian
<a href="/">Blog</a> in WordPress but I’d like to use Jekyll and this is a
good opportunity. Share my experience and my Job, to grow and improve my
skills! Can you help me with my English? :P</p>
<h3 id="skills-and-interests">Skills and interests</h3>
<p>This is a list of my skills and interests that I am sure will be all topics for
my posts, I hope you will enjoy reading them! Php, tech, html, css, js, Open
Source, Zf2, Doctrine, Symfony, noSql (mongo, couch..), sql databases, Redis,
DevOps, Chef, Vagrant, Composer, TDD….</p>
<h3 id="open-source-world">Open Source world!</h3>
<p>Community and Open Source are my passion! A community is a great way to
challenge myself as a person and as a coder. You can find me on <a href="https://github.com/gianarb">Github
Account</a>!</p>
<h3 id="this-is-my-face">This is my face!</h3>
<div style="text-align:center;">
<img src="/img/posts/2013-07-19-why-this-blog.png" width="90%" />
</div>