MySQL Replication: What’s New in MySQL 5.7 and Beyond

Very interesting presentation about current and upcoming features of replication in MySQL 5.7.x:

MySQL Replication: What’s New in MySQL 5.7 and Beyond from Andrew Morgan

Semi-irregular Sysadmin Ninja's Github Digest (Vol. 15)

Hi All,

Let me introduce you Volume 15 of Sysadmin Ninja's Github Digest! (as usual, in no particular order).
But why volume 15, if previous one was 13 and where's 11 and 14 then?
Well, because 11 and 14 are still in my drafts, and although I'm doubt that I'm publish them sometimes... well, let's be at least consistent - I'll try to continue digest publishing (will try to do it) on weekly basis.

1. KeyBox.
A web-based SSH console that executes commands on multiple shells. KeyBox allows you to manage keys, share terminal commands, and upload files to multiple systems simultaneously. http://sshkeybox.com
https://github.com/skavanagh/KeyBox

"A web-based ssh console to execute commands and manage multiple systems simultaneously. KeyBox allows you to share terminal commands and upload files to all your systems. Once the sessions have been opened you can select a single system or any combination to run your commands. Additional system administrators can be added and their terminal sessions and history can be audited. Also, KeyBox can manage and distribute public keys that have been setup and defined."

Can be useful for small distributed team of sysadmins across the globe.

2. FNordmetric
FnordMetric allows you collect and visualize timeseries data with SQL. http://fnordmetric.io
https://github.com/paulasmuth/fnordmetric

Hot stuff. Client-server application which "aims to be a StatsD+graphite competitor, it implements a wire compatible StatsD API". The main idea behind it that you write your query in SQL-like language, called "ChartSQL" and get your graph in SVG format. For it has quite big amount of graphing modes, but almost lacks of any functions for now. Written in C++, so, quite fast, but good luck with porting Graphite function library. Maybe it's a good idea to port Graphite as backend for it. Looks promising anyway.

And talking about Graphite -
3. graphite-stresser
A stress testing tool for Graphite
https://github.com/feangulo/graphite-stresser

Nothing unusual, but nice tool if you want to stress test your Graphite instance. Check author blog's entry for details.

4. pstop
pstop - a top-like program for MySQL
https://github.com/sjmudd/pstop

"pstop is a program which collects information from MySQL 5.6+'s performance_schema database and uses this information to display server load in real-time." For example, you can get IOPS for innodb file, or locks / operations / latencies per table. At least, you can start using performance schema in your MySQL 5.6 instance for something useful. Another useful P_S tool is "sys schema", you can read recent entry in Percona blog about it.

5. Consul-template
Generic template rendering and notifications with Consul
https://github.com/hashicorp/consul-template

Quite recent addition to service discovery tool Consul.io - now you can use it for any service which didn't understand service discovery through DNS - you can format config file templates for that service and reload it when Consul will see configuration change. Agian, blog post is better than 1000 words.

6. VCLfiddle
VclFiddle is hosted at http://www.vclfiddle.net/
https://github.com/vclfiddle/vclfiddle
"VclFiddle is an online tool for experimenting with the Varnish Cache HTTP reverse-proxy in a sandboxed environment. The name comes from a combination of the Varnish Configuration Language (VCL) and another tool that inspired this project, JSFiddle."

I.e. you can edit your VCL config on-line, using web editor, and check how it caches your website.

7. Racher.io
Rancher is an open source project that provides infrastructure services designed specifically for Docker. http://www.rancher.io
https://github.com/rancherio/rancher

Quite ambitious project for creating AWS-like environment, but for Docker containers. 

8. Atlas
A high-performance and stable proxy for MySQL
https://github.com/Qihoo360/Atlas

Another MySQL proxy. Well... Personally I never saw any production running on some MySQL-proxy solution (even on MySQL Fabric), but some China company named Qihoo360 developed this solution and insists that it's running on their production infrastructure.

9. Bosun
An advanced, open-source monitoring and alerting system by Stack Exchange http://bosun.io
https://github.com/bosun-monitor/bosun

Another Graphite competitior - OpenTSDB-backed service with own system metric's collector scollector and graphing and alerting interface. Written in Go. Looks neat and scalable:

And speaking about Go -

10. Go-opstocat
Collection of Ops related patterns for Go apps at GitHub.
https://github.com/github/go-opstocat

and
11. Delve
Delve is a Go debugger, written in Go.
https://github.com/derekparker/delve

Going further.

12. bup
Very efficient backup system based on the git packfile format, providing fast incremental saves and global deduplication (among and within files, including virtual machine images).
https://bup.github.io/
https://github.com/bup/bup

Interesting new backup tool, quite green and fresh, but looks promising.

13. osquery
SQL powered operating system instrumentation, monitoring, and analytics.
http://osquery.io
https://github.com/facebook/osquery

Facebook quite recently open-sourced that tool. Idea is looks very promising - again present system state as SQL tables on which you can run queries, by interactive console or automatically, as daemon.

And fun section

15. C4
C in four functions
https://github.com/rswier/c4

C-compile in 500 lines of C. Reading of it's sources is quite fun. :)

16. Gravity
An orbital simulation game written in Elm
https://github.com/stephenbalaban/Gravity

You can play it here.

17. Convergence
Python/OpenCl Cellular Automata design & manipulation tool
https://github.com/InfiniteSearchSpace/PyCl-Convergence

Looks like fun, but can't run that on my Mac for some reason, so, no screenshots.

End of MySQL vs PostgreSQL arguments

Russian text is here.

Good article about MySQL vs Postgresql. Summary for lazy readers -
"MySQL is designed with the idea that applications provide logic and the database provides dumb storage of the application's state.  While this has changed a bit with the addition of user-defined functions and stored procedures, the overall design constrains MySQL primarily to this use case.  This is not necessarily a bad thing as, traditionally, software licensing costs and requirements have often required that even advanced database systems like Oracle are used in this way.  MySQL targets the "my app, my database" world and is usually sufficient for this, particularly when lowest common denominators are used to ensure portability.

PostgreSQL, on the other hand, is designed with the idea that the database itself is a modelling tool, and that the applications interact with it over an API defined in SQL.  Object-relational modelling advocates point out that often getting acceptable performance in complex situations requires an ability to put some forms of logic in the database and even tie this to data structures in the database.  In this model, the database itself is a development platform which exposes API's, and multiple applications may read or write data via these API's.  It is thus best seen as an advanced data modelling, storage, and centralization solution rather than as a simple application back-end.

These differences show, I think, that when PostgreSQL people complain that MySQL is not a "real database management system" and MySQL people dispute this that in fact the real difference is in definitions, and in this case the definitions are deceptively far apart.  Understanding those differences is, I think, the key to making an informed choice." 

Sad news, camrades

Russian text is here

Sad news, comrades.
In all holywars about Perl vs Everything some high-loaded Perl projects was already mentioned. One of them was YouPorn - one of largest porn ahem .. adult entertainment websites in the world.
But not anymore.
You can check presentation (PDF / PPTX / GoogleDocs) of their technical lead on Confoo conference -

• Written in PERL with a very complex architecture
• First few months dedicated to learning the site, maintain it, and plan the re-write.
• Re-write started in August 2011 and was originally planned for a delivery in mid-November.
• Actually launched at the end of January.

So, in the end of 2011 site was completely rewritten on PHP - it's simplier, faster and easier to find staff.
Another architecture is pretty neat and straightforward -  HaProxy + Varnish + Nginx/PHP-FPM/Symfony2 + Redis/MySQL + Syslog-ng for logging, only one strange decision - they are using ActiveMQ for DB/Redis writes managing, but as far as I understand from presentation it's not very good solution for them...