Semi-irregular Sysadmin Ninja's Github Digest (Vol. 21)

Hello, fellow readers!
Issue 21 of "Semi-irregular Sysadmin Ninja's Github Digest" is here. The last issue was very dry, will add more of my thoughts and funny pictures. :)
Let's go!


teeproxy
"A reverse HTTP proxy that duplicates requests."
"You may have production servers running, but you need to upgrade to a new system. You want to run A/B test on both old and new systems to confirm the new system can handle the production load, and want to see whether the new system can run in shadow mode continuously without any issue."
https://github.com/chrislusf/teeproxy

kicksat
"A tiny open source spacecraft project. http://kicksat.io"

WOW. Just W-O-W. Your eyes are not lying, it's open-source spacecraft. "Our goal is to dramatically lower the cost of spaceflight, making it easy enough and affordable enough for anyone to explore space. We can do this by shrinking the size and mass of the spacecraft, allowing many to be launched together."

I hope guys will be succeeded and we'll see KickSat launch soon!
https://github.com/kicksat

web2web
"P2P web powered by torrents and blockchain."
Rejoice, my paranoid brothers and sisters! New Internet is here! Wear our foil hats on!
It's a combination of webtorrent and blockhain to make not-seizable internet!
"When you open index.html in the browser (live demo), here's what happens:
Bitcoin address 1DhDyqB4xgDWjZzfbYGeutqdqBhSF7tGt4 is searched for the latest outgoing transaction containing OP_RETURN script. Inside the script there is a torrent infohash of webpage.html. webpage.html is downloaded from torrent via webtorrent and displayed."
https://github.com/elendirx/web2web

ironssh
"IronSSH - End-to-end secure file transfer"

"While sftp and scp use ssh to keep files secure while they are being transferred over the network, once those files hit the remote server, they are no longer protected. The ironsftp executable provides additional security. When you put a file on the server using ironsftp, the file is encrypted before it is uploaded, and it stays that way on the server. When you get a file from the server, it is downloaded then decrypted. So the file remains secure until it is at the place you want to use it - on your local machine."
https://github.com/ironcorelabs/ironssh

quinedb
"QuineDB is a quine that is also a key/value store.
If your database can't print its own source code, can you really trust it?"Very interesting and funny project! It's simple K/V storage, written in bash4, but it's also a quine!
"When you run it, the (possibly modified) source code of quinedb is printed to STDOUT, and the results of the specific command run are printed to STDERR."
https://github.com/gfredericks/quinedb


flask_jsondash
"Build javascript chart dashboards without any front-end code. Uses any json endpoint. JSON config only. Ready to go."
Long time dream is fulfilled! Yes, dashboards w/o any front-end code. !https://github.com/christabor/flask_jsondash


lograil
"LogTrail is a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by Papertrail."
Like "tail -f", but for ELK!

https://github.com/sivasamyk/logtrail

python-remote-companies

"A list of companies that allow remote work and use Python."
Yep, that simple, but maybe useful.https://github.com/mariusavram91/python-remote-companies

Games on the Github 
"list of open source games and game-related projects that can be found on GitHub"
https://github.com/leereilly/games

cog
"Bringing the power of the command line to chat http://operable.io"
"Cog is an open chatops platform that gives you a secure, collaborative command line right in your chat window. It is designed to be secure, highly available, chat provider agnostic, and to be extensible using your favorite programming language."
https://github.com/operable/cog


borg
"BORG - A terminal based search engine for bash commands"

Searching bash-related questions on Stackoverlow not leaving your terminal!
https://github.com/crufter/borg

git-stacktrace
"Easily figure out which git commit caused a given stacktrace https://pypi.python.org/pypi/git-stacktrace"

A little bit naive tool which helps you to find out wich commit is responsible for specific stacktrace. Python and Java are supported.
https://github.com/pinterest/git-stacktrace

pyinfra
"⚡ Deploy stuff by diff-ing the state you want against the remote server"
Interesting deploy tool. Looks nice, but IMO it better uses real configuration management tool in this case, e.g. Salt or Ansible.
https://github.com/Fizzadar/pyinfra

And something more for ML fans:

deepregex
"Neural Generation of Regular Expressions from Natural Language with Minimal Domain Knowledge"
For now, it's purely academic project - creating regexes using natural language and machine learning.
But beware - regexes it's a start, maybe sometimes computers maybe be able to write own programs.
Why they will need people then? :)
https://github.com/nicholaslocascio/deep-regex

tensorflow_image_classifier
Machine Learning is simple! You can make own TF image classifier in 5 minutes!

https://github.com/llSourcell/tensorflow_image_classifier

Semi-irregular Sysadmin Ninja's Github Digest (Vol. 18)

I was slacking for a long time, I know. Sorry for that. I'll push two issues in a row now, this is the first one. Will try to make it more regular, will include other sources too.

1. The Crystal Programming Language 
http://crystal-lang.org
https://github.com/manastech/crystal
New programming language, named Crystal. "We love Ruby’s efficiency for writing code. We love C’s efficiency for running code. We want the best of both worlds." Programs look like Ruby, but compiles to efficient native code, and has compile-time error evaluation like Rust. Worth to check out, if you're PL freak, like me. :)

2. chef-koans
An experimental, test-driven way to learn about Chef.
https://github.com/leftathome/chef-koans
"An experimental, test-driven way to learn about Chef. Takes some inspiration from Ruby Koans and from other things that are awesome and simple." Unfortunately, only lesson number 0 is ready now - but you're welcome to contribute, of course!
Also, if you didn't read Vim koans or Git koans - please try, it's quite fun.

3. node-bell
Real-time anomalies detection for periodic time series.
http://eleme.io/blog/2014/metrics-monitor
https://github.com/eleme/node-bell

4. streem
prototype of stream based programming language
https://github.com/matz/streem
A prototype of new PL from an author of Ruby - Yukihiro "matz" Matsumoto. It's on very early stage of development.

5. sfs
Asynchronous Filesystem Replication
https://github.com/immobiliare/sfs

6. gitfs
Version controlled file system
http://presslabs.com/gitfs/
https://github.com/PressLabs/gitfs

7. awesome-courses
List of awesome university courses for learning Computer Science!
https://github.com/prakhar1989/awesome-courses

8. mochi
Dynamically typed functional programming language
https://github.com/i2y/mochi

9. consul-do
Do something based on leadership status
https://github.com/zeroXten/consul-do

10. openbay
The Pirate Bay source code.
https://github.com/isohuntto/openbay

Semi-irregular SysadminNinja's Github Digest (vol. 13)

Hi All,

Third part of digest, most current one!
Two more to come.

1. gogs
Gogs(Go Git Service) is a painless self-hosted Git Service written in Go.
http://gogs.io
https://github.com/gogits/gogs

Another Github clone, but written in Go. Worth checking, if you need one.

2. pyston
An open-source Python implementation using JIT techniques.
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/
https://github.com/dropbox/pyston

New Python implementation, blazingly fast, looks promising, recommended by Guido himself :).

3. profiling
An interactive Python profiler.
https://github.com/what-studio/profiling

Nice looking Python profiler, still under development but looks good:

4. blackbox
Safely store secrets in Git/Mercurial http://the-cloud-book.com
https://github.com/StackExchange/blackbox

Another solution for storing secrets in your repos, but now from StackExchange and @yesthattom himself. Fresh and professional looking.

5. sfr1-lite
Search Formula-1 - A distributed high performance massive data engine for enterprise/vertical search
https://github.com/izenecloud/sf1r-lite

Vertical search engines are quite rare in open-source landscape. It's a new one from some Chinese startup, check tech docs in English for details.

6. mannaggia
automatic saint calling for depressed Veteran unix Admins, in italian
https://github.com/LegolasTheElf/mannaggia

Hah, funny thing. "Automatic saint calling for depressed Veteran unix Admins, in italian
Developed in italian, can be easily adapted in other languages."
TIL what "mannaggia" means. :)

7. shellshoc_poc
Shellshock DHCP RCE Proof of Concept
https://github.com/mschwager/shellshock_pocProof of concept of shellshock attack on DHCP server. See https://www.trustedsec.com/september-2014/shellshock-dhcp-rce-proof-concept/ for details:

8. bash_shellshock 

Wrapper for /bin/bash that mitigates 'shellshock' 

https://github.com/dlitz/bash-shellshock 

If you do not trust current Shellshock fixes you can use own wrapper to secure your server(s).

Weekly Ops Github Digest #8 (July 29th 2014)

So, it seems that our digest becoming quite regular, it's issue number 8 and number 7 was out exactly one week ago, same as issue 6 was two weeks ago, nice.

Let's try to keep it that way - new issue every Monday/Tuesday, for fresh start of new week!
... but please note that possible upcoming vacation can interfere with my intentions. :)

OK, let's go.

1. elvish
An experimental Unix shell
https://github.com/xiaq/elvish
Nice new experimental shell (written in Go), more traditional than Xiki but also with intresting new features, like nice autocompletion and navigation mode, built in daemon for syncing command history from different instances and new Clojure/Lisp-like built-in language. Worth a try, but only if you have Linux, it doesn't compile on Mac, so, if you have one and want to be a contributor - welcome.

2. perf-tools
Performance analysis tools based on Linux perf_events (aka perf) and ftrace.
https://github.com/brendangregg/perf-tools
Set of performance analyzing tools for Linux, using (relatively) new Ftrace interface, written by famous Brendan D. Gregg, author of Systems Performance book, DTrace book etc.

Will not talk much about that, in short - amazing stuff, check my previous post about it.

3. Next 3 repos connected with Docker, lets merge them together:

Fig.
Fast, isolated development environments using Docker http://www.fig.sh
https://github.com/orchardup/fig
Describe your environment in single file and run it with single command "fig up".
Looks cool? Then check out

Octohost.
Simple web focused Dockerfile based PaaS server.
http://www.octohost.io/
https://github.com/octohost/octohost
Similar idea, but realization is little bit different - and idea is more general and limited at the same time. Just add Dockerfile to your webapp and after that run "git push" to deploy your project on choosen environment (Vagrant/Amazon/Digital Ocean/Rackspace/own server) - but only limited (but quite poular) languages and platforms are supported.

Next thing -
Registrator.
Service registry bridge for Docker
https://github.com/progrium/registrator
will help you with automatic "register/deregisters services for containers based on published ports and metadata from the container environment". I.e. you can automatically add/remove services to LB, for example. External service registers/pairing/discovery are supported also. Nice thing if your building scalable application on Docker.


Going to more general developer's stuff.

4. Two Git help pages. More general

Git Cheat Sheet
https://github.com/ArslanBilal/Git-Cheat-Sheet

and more specific

git-flight-rules
https://github.com/k88hudson/git-flight-rules

First one is normal Git cheat sheet, but second one is more like FAQ and list of common tasks and workarounds.


5. CockroachDB.
A Scalable, Geo-Replicated, Transactional Datastore
https://github.com/cockroachdb/cockroach

So, it looks like NoSQL is not very popular buzzword now, NewSQL is much better!
But if talking seriously it really looks promising - it took better parts from both worlds:



ALPHA for now, we'll see how it'll going.

6. pycrastinate
TODO less, DO more. Keep your code clean without changing the way you code.
https://github.com/isaacbernat/pycrastinate
Pycrastinate is a language-agnostic tool that helps you keep your codebase (whether it is legacy or new) under control in a transparent way (i.e. without interfering with your coding). I.e. it checks for FIXMEs and TODOs and gently tring to force you to do something with that. :)
Check this talk or slides if interested.

7. Streisand.
https://github.com/jlund/streisand
"Streisand sets up a new server running L2TP/IPsec, OpenSSH, OpenVPN, Shadowsocks, Stunnel, and a Tor bridge. It also generates custom configuration instructions for all of these services. At the end of the run you are given an HTML file with instructions that can be shared with friends, family members, and fellow activists."
Not like very funny thing, especially for persons who struggle with internet censorshop somewhere but worth to checking then.

And time for fun!

8. IPFS
The Permanent Web http://ipfs.io
https://github.com/jbenet/ipfs
According to website:
"IPFS is a global, versioned, peer-to-peer filesystem. It combines good ideas from Git, BitTorrent, Kademlia, SFS, and the Web. It is like a single bittorrent swarm, exchanging git objects. IPFS provides an interface as simple as the HTTP web, but with permanence built in. You can also mount the world at /ipfs."
Very experimental thing for now, but maybe it's the source of new Internet, who knows?

Sporadic Ops GitHub digest #3 (02 Jul 2014)

We are still rollin'. Go!

1. Tessera.
A dashboard front-end for graphite 

https://github.com/urbanairship/tessera
Good and promising Graphite dashboard.

2. BitAuth
Authenticate with web services utilizing the same strategy as Bitcoin.
https://github.com/bitpay/bitauth
Proposal for a new passwordless authentication protocol which is not storing user passwords at all. :)
See their blog for details.

3. cheap secrets
Store your passwords in git
https://github.com/bpo/cheap-secrets
Personal password storage in Git, nice. PGP encrypted, of course.

4. doitlive
doitlive is a tool for live presentations in the terminal. It reads a file of shell commands and replays the commands in a fake terminal session as you type random characters.
https://github.com/sloria/doitlive
If you need to do presentations of your terminal sessions.

5. Ambassadord - A Docker ambassador (containerized TCP reverse proxy / forwarder) that supports static forwards, DNS-based forwards (with SRV), Consul+Etcd based forwards, or forwards based on the connecting container's intended backend (read: magic).
https://github.com/progrium/ambassadord
Too much to explain. :) Read about Docker first.