Skip to content

Category Archives: Technology

Things Technological

Four Questions For Engineers

One way of looking at the art and science of software engineering is that it is a process of mapping human desires and wishes — the insides of peoples’ heads — onto a computer system. This is not a particularly novel idea, and it’s one that you are probably familiar with, but it’s an important one. Engagement with a […]

Cross-Account use of AWS CLI

The documentation around using the AWS CLI from an AWS EC2 instance on one account to access resources in another account are not great. The information is all there, somewhere, but it’s scattered across many places and to derive what you need from those sources you have to pretty well read all the sources. Two […]

Oh no! The certificate has expired!

Hey kids! You know those SSL certificates you obtained and installed today? Yeah, put a reminder in your calendar right now for a week before the expiry date, so you don’t get caught out. Future you will thank you.

OpenSSL on HighSierra

Recently I finally got around to reading the excellent OpenSSL Cookbook from Ivan Ristić – you can grab a free copy via https://www.openssl.org/docs/ – and the first question in my mind was “what version of OpenSSL is already installed on my Mac”. A quick check showed it’s there pre-built in HighSierra in /usr/bin: Hmm. Wikipedia […]

TLS 1.3 – It’s like Christmas

Via The Register I see that TLS 1.3 has finally rolled off the standards and committee draft assembly line. This is pretty big news, not least because we’ve been working with the current TLS 1.2 standard for almost a decade, and the defects in it have well and truly been discovered and exploited. There’s a […]

Bootstrapping AWS with Terraform and CodeCommit

A rough model that I’ve been working on and thinking about recently is for the AWS account (or accounts) be put together so that there’s a “bastion” or “bootstrap” instance that can be used to build out the rest of the environment. There is a certain chicken-and-egg around this, particularly if you want to use […]

Creating a custom Kylo Sandbox

I had a need – or desire – to build a VM with a certain version of NiFi on it, and a handful of other Hadoop-type services, to act as a local sandbox. As I’ve mentioned before, I do find it slightly more convenient to use a single VM for a collection of services, rather […]

Smoke testing Kafka in HDP

Assuming that you have a vanilla HDP, or the HDP sandbox, or have installed a cluster with Ambari and added Kafka, then the following may help you to smoke test the behaviour of Kafka. Obviously if you’ve configured Kafka or Zookeeper to be running on different ports, this isn’t going to help you much, and […]

Lies, Damned Lies and Programmers

I recently came across a really nice set – not directly related – of articles dealing with various profound errors that programmers and system designers fall into when dealing with names and addresses. The TL;DR if you don’t read these: names and addresses are hard and most things you believe about them are wrong. Let’s […]

A Demonstration NiFi Cluster

In order to explore NiFi clustering, and NiFi site-to-site protocol, I decided that I could use a minimal installation – as I’m really just exploring the behaviour of NiFi itself, I don’t need to have any Hadoop environment running as well. To this end, my thought was that I could get the flexibility to just […]