Dr StrangeBot: Or How I Learned to Stop Worrying and Trust Machine Learning

This post was originally published as part of my role at WhiteHat Security Links have been added for context/comedy/my own entertainment, but no content has been modified Beneath the cynicism, hyperbole, market–making and FUD; the strategic importance of AI in Cybersecurity is only constrained by us ‘meatbags’. Being a data science practitioner in the cybersecurity space has been a double–edged sword for several years. On the one hand, with the proliferation of automated security testing, network IDS advances, the sheer growth in traffic and the threat surface of our increasingly complex, interconnected application development practices, these roiling oceans of flotsam and datum are everything our data hungry little hearts desire. Related innovations in data engineering in the past decade mean that questions that had previously only lived in the craven dreams of executive officers and deranged analysts are now the kind of tasks that we hand off to interns to make sure they have correctly set up their workstations. ...

March 24, 2021 · Andrew Bolster

A Stranger in a Strange Land: Data Science Onboarding In Practice

This talk was originally prepared for the 2020 Northern Ireland Developers Conference, held in lockdown and pre-recorded in the McKee Room in Farset Labs Intro Data Science is the current hotness. While those of us in these virtual rooms may make fun of the likes of Dominic Cummings for extolling a ‘Data Driven Approach’ to policy, the reality is that Data Science as a buzzword bingo term has survived and indeed thrived in a climate where ‘Artificial Intelligence’ is increasingly derided as being something that’s written more in PowerPoint than Python, ‘Machine Learning’ still gives people images of liquid metal exoskeletons crushing powdery puny human skulls, and those in management with long memories remember what kind of mess “Quantitative Analysis” got us into not too long ago… ...

October 20, 2020 · Andrew Bolster

Tell me about your Programmer - Robopsychologist and other careers that don't exist (yet)

This talk was originally prepared for NI Raspberry Jam’s Kids Track, associated with the full Northern Ireland Developers Conference, held in lockdown and pre-recorded in the McKee Room in Farset Labs In Issac Asimov’s stories, the technical, social and personal impacts of advanced robotics and artificial intelligence are explored. One creation in his books was the career of “Robopsychologist”, a combination of mathematician, programmer, and psychologist, that diagnosed and treated misbehaving AI. In this talk we’ll discuss how on earth you can prepare for careers in Robopsychology and other careers that don’t exist (yet). ...

October 19, 2020 · Andrew Bolster

Back to reality

Realised it’d been a while, thought it was time for an update. I’m gonna try and make an effort to get closer to an ideal of something like #weeknotes as I’ve always been jealous of Adrian’s ability to put this thoughts down and get them out the door in a sensible way… Anyway, use the Table of Contents above to skip to bits you may care about. Farset Updates So, this was the actual reason for this update but it started a weird thought process that led to this blog. ...

September 8, 2020 · Andrew Bolster

Merging Git Repos for Archival Purposes

TL;DR I had reason to want to combine git repos into one big repo consisting of repos in their own folders, while ideally maintaining the histories of all those repos for archaeological purposes. There are many reasons why someone would want to do this, and my specific use case isn’t relevant. Good luck. Why so complicated? ‘Hidden’ files (dotfiles) suck Shell Wildcards suck Wildcards with selective exclusions (i.e. .git) suck File names with spaces suck Trailing Slashes suck Rewriting History sucks Raw version because I don’t trust GIST and embeds and such…. #!/bin/bash usage() { cat << EOF This script imports a git repo (accessible from https://\$origin/\$user/\$repo) and all its history as subdirectory of a destination (available locally at \$dest) It is designed for non-production, archival processes and may destroy everything you've ever loved because you looked at it funny. You have been warned. The structure of the destination will end up something like this: ~/src - \$dest - origins - \$origin - \$user - \$repo Required Arguments: -u|--user: The user that owns the repo to be imported -r|--repo: The name of the repository to be imported -d|--dest: The local name of the destination repository (assumed to be under ~/src) -o|--origin: The git server that is the origin of the repo to be imported EOF } if [ $# -le 6 ]; then usage exit 1 fi while [[ "$#" -gt 0 ]]; do case $1 in -u|--user) user="$2"; shift ;; -r|--repo) repo="$2"; shift ;; -d|--dest) dest="$2"; shift ;; -o|--origin) origin="$2"; shift ;; *) echo "Unknown parameter passed: $1"; usage; exit 1 ;; esac shift done tmp="/tmp/_${dest}_tmp" echo "Importing $origin/$user/$repo into $dest" rm -rf ~/src/$repo cd ~/src git clone https://$origin/$user/$repo cd $repo git filter-branch \ --tree-filter "mkdir -p $tmp/origin; git ls-files | cpio -pdumB $tmp/origin; git ls-files | xargs -d '\n' rm -r; find . -type d -empty -delete; mkdir -p origins/$origin/$user; mv $tmp/origin origins/$origin/$user/$repo/"\ --tag-name-filter cat --prune-empty -- --all if [ $? -eq 0 ]; then ## WAIT PATIENTLY cd ../$dest git remote add $repo ../$repo git fetch $repo --tags git merge --allow-unrelated-histories $repo/master #Youre on your own if you want a different / multiple branch(es)... git remote remove $repo else echo failed for $user/$repo fi

June 23, 2020 · Andrew Bolster

Pragmatic Data Science; When Unstoppable Math meets Immovable Ethics

This is a Rough Transcript from The Virtual Bash on Ethics Introduction Around two thousand, two hundred and 20 years ago, Archimedes said “Give me a place to stand and I will move the world”. Archimedes has come to be (rightly) associated with many fundamental innovations in mathematics and engineering, and while I’m sure we’re all sick and tired of screws and levers and jumping out of bathtubs, I’d like to start off with a discussion on one of his more mundane creations, the compound pulley. ...

May 22, 2020 · Andrew Bolster

Is Your AI Ethical?

Originally posted in RTInsights Businesses should do their part to ensure products are designed judiciously to reflect core company values and provide audit trails of how AI is learned. As we examine an increasing reliance on artificial intelligence (AI) and machine learning, it’s being revealed that AI can have a built-in bias, whether intentional or not. In late 2019, Apple and Goldman Sachs faced allegations that the Apple Card used an algorithm that discriminated against women in credit-scoring evaluation – after Apple’s own co-founder Steve Wozniak and entrepreneur David Heinemeier Hansson received credit limits of 10-20 times higher than their wives. ...

April 26, 2020 · Andrew Bolster

The Importance of Active Learning in Data Science and Engineering

Originally posted in Cybersecurity Insiders Back when I was pursuing my undergraduate degree in electronics and software engineering, I couldn’t imagine a path that would lead to me working with NATO on port protection and maritime defense, teaching smart submarines how to trust each other. But while I was working toward a Ph.D., that’s what happened. Instead of following the path into academia, a friend enticed me to work with him on biometrics. From there, I found an opportunity to apply my skills and knowledge to the cybersecurity industry – but that’s not something I could have predicted either. ...

March 4, 2020 · Andrew Bolster

Mosquitto (MQTT) Emon Pi (Open Energy Monitor) Forwarding Bridge

Super quick one this time; I’ve been experimenting with MQTT to act as a central messaging broker for “Farset In-Space Related Stuff” as part of the near continuous renovations and expansions. We previously had a well configured EMonPi set up with nice dashboards and things, but that died a death at some point during the move, who knows. Anyway, EmonPi has a built in mosquitto broker, which it uses to keep ‘state’ across several parts of the emonpi ecosystem. ...

February 18, 2020 · Andrew Bolster

Headless Pi Configuration with Multi-Wifi Remote Access

Raspberry Pi’s are great, but often have a lot of baggage associated with them, and I keep forgetting all the clever things you can do to get them up and running without having a sea of cables attached… Results Raspberry Pi that can be sshd into from anywhere in the world* without poking any firewall rules or anything other than power connected to it, that works in a range of WiFi access points. ...

February 17, 2020 · Andrew Bolster