Regarding one of my latest post, talking about Fluentd, some of you asked me why I’ve chosen Fluentd instead of Logstash. First of all, I’ve looked at this blog post, which kindly resume the current situation. Looking at this and with feedback I got, I could establish this array: <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #c9daf8; font-weight: bold; text-align: center;" data-sheets-value="[null,2,&quot;Logstash&quot;]"> Logstash </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #c9daf8; font-weight: bold; text-align: center;" data-sheets-value="[null,2,&quot;Fluentd&quot;]"> Fluentd </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; text-align: center;" data-sheets-value="[null,2,&quot;Flexibility/Interoperability&quot;]"> Flexibility/Interoperability </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; text-align: center;" data-sheets-value="[null,2,&quot;Simplicity/Robustness&quot;]"> Simplicity/Robustness </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; text-align: center;" data-sheets-value="[null,2,&quot;JRuby&quot;]"> JRuby </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; text-align: center;" data-sheets-value="[null,2,&quot;Cruby&quot;]"> Cruby </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;More&quot;]"> More </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #f4cccc; text-align: center;" data-sheets-value="[null,2,&quot;Fewer&quot;]"> Fewer </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #f4cccc; text-align: center;" data-sheets-value="[null,2,&quot;Basics&quot;]"> Basics </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;Scalables&quot;]"> Scalables </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #f4cccc; text-align: center;" data-sheets-value="[null,2,&quot;Lot&quot;]"> Lot </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;Lot more&quot;]"> Lot more </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;Simple (drop a jar)&quot;]"> Simple (drop a jar) </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #fff2cc; text-align: center;" data-sheets-value="[null,2,&quot;Can be complex (dependancies)&quot;]"> Can be complex (dependencies) </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;Yes&quot;]"> Yes </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;Yes&quot;]"> Yes </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #f4cccc; text-align: center;" data-sheets-value="[null,2,&quot;Forward to 2 hosts&quot;]"> Forward to 2 hosts </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;LB A/A or A/P&quot;]"> LB A/A or A/P </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #f4cccc; text-align: center;" data-sheets-value="[null,2,&quot;Can loss messages&quot;]"> Can loss messages </td> <td style="padding: 2px 3px 2px 3px; vertical-align: bottom; background-color: #d9ead3; text-align: center;" data-sheets-value="[null,2,&quot;No message loss&quot;]"> No message loss </td> Logstash and Fluentd are both powerful solutions, they both got pros and cons.

Continue reading

Managing logs is not a complicated tasks with classical syslog systems (syslog-ng, rsyslog…). However, being able to search in them quickly when you have several gigabit of logs, with scalability, with a nice graphical interface etc…is not the same thing. Hopefully today, tools that permit to do it very well exists, here are the list of tools that’s we’re going to use to achieve it: Elasticsearch: Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Continue reading

As you may know, I’m using and loving MariaDB. A colleague recently introduced me the benefits of the php built-in MySQL/MariaDB library (shame on me, I never heard of it). Here are the informations you can find from it on the MySQL website: The mysqlnd library is highly optimized for and tightly integrated into PHP. The MySQL Client Library cannot offer the same optimizations because it is a general-purpose client library.

Continue reading

On next Monday, a meetup will take place in Paris concerning Big Data with Twitter feeback. This meetup will be live retransmitted through YouTube. The main talk will speak about how Twitter did to scale to millions of users: Twitter is one of the most heavily trafficked sites on the internet, with over 240 million active users and serving over 500 million Tweets per day. Twitter, as a company and as a service has seen remarkable growth in its eight years of existence.

Continue reading

My last post was about existing alternatives to Dropbox. I also mentioned that I was trying Seafile. I’m running now Seafile since a few days now and it works without any issues. The Linux and Android clients works very well. I just hope a sync feature in the Android version will appear. Regarding the web version, there are all attending collaboration features and that’s great. Seafile supports file versioning and trash bin.

Continue reading