Welcome!

Enterprise DevOps, Log Management and Analytics

Sematext Blog

Subscribe to Sematext Blog: eMailAlertsEmail Alerts
Get Sematext Blog via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Latest Blogs from Sematext Blog
Recently, a few people from Sematext’s Logsene team debated about how useful the “structured” part of syslog logs (those using the RFC5424 format) is to people.  Or has shipping logs in other structured formats, like JSON, made RFC5424 irrelevant to most people?  Here...
Good news for Docker fans: we’re running a third Docker Monitoring webinar on Tuesday, October 6 at 2:00 pm Eastern Time / 11:00 am Pacific Time. If you use Docker you know that these deployments can be very dynamic, not to mention all the ways there are to monitor Docker contain...
Upgrading to the new major version right after its release without waiting for the inevitable .1 release Remembering that you said, “We don’t need backups, we have shard replicas” to your manager during an 8-hour cluster recovery session Not running dedicated masters and wondering why ...
This is a post of the work done at BloomReach on smarter index & data management in SolrCloud.   Authors: Nitin Sharma – Search Platform Engineer & Suruchi Shah –  Engineering Intern   Introduction In a multi-tenant search architecture, as the size of data grow...
Akka is a toolkit and runtime for building highly concurrent, distributed and resilient message-driven applications on the JVM. It’s a part of Scala’s standard distribution for the implementation of the “actor model”. How Akka Works Messages between Actors are exchanged in Mailbox queu...
Upgrading to the new major version right after its release without waiting for the inevitable .1 release Explaining your, “I don’t need backups, I can always reindex” statement to your manager during an 8-hour reindexing session Taking down the whole Data Center with a single rows=1000...
In the dynamic world of “Microservices” the traditional method of static setups for log routing and parsing doesn’t work very well; in fact, it creates additional complexity and resource usage.  This, in turn, reduces the number of microservices that could run on a single server.  Sema...
If you use Docker you know that these deployments can be very dynamic, not to mention all the ways there are to monitor Docker containers and their hosts, collect logs from them, etc. etc.  And if you didn’t know these things, well, you’ve come to the right place! With this pair of ide...
Today we’re happy to officially introduce AppMaps. What’s AppMap? As you can see below, AppMap is a map-like visual representation of your complete application architecture. AppMaps show which components are communicating with which components, at what throughput and latency, at what n...
When you’re building a monitoring solution or evaluate existing ones, what do you look for? Probably these four core aspects of functionality: Collection and display of metrics Alerting based on metric values and anomalies Collection and display of server and application logs and ...
Stage setting: Camera is positioned above the treetop of one of three tall poplars. It looks down on the terrace of a pub. It’s evening, but there’s still enough light to see that the terrace is sparsely populated. Camera slowly moves down towards a specific table in the corner… As t...
If you’ve got an interest in things like CoreOS, logs and monitoring then you should check out our previous CoreOS-related posts on Monitoring Core OS Clusters and how to get CoreOS logs into ELK in 5 minutes. And they are only the start of SPM integrations with CoreOS! Case in point:...
There are many ways to skin a cat. If you use Docker you know that these deployments can be very dynamic, not to mention all the ways there are to monitor Docker containers, collect logs from them, etc. etc.  And if you didn’t know these things, well, you’ve come to the right place! Se...
[Note: This post is part of a series on Transaction Tracing — links to the other posts are at the bottom of this post] If you missed the Distributed Transaction Tracing Intro, here’s the key bit you should know: Distributed Transaction Tracing is great for: Pinpointing root cause...
We have been using Node.js here at Sematext and, since eating one’s own dogfood is healthy, we wanted to be able to monitor our Node.js apps with SPM (we are in performance monitoring and big data biz). So, the first thing to do in such a case is to add monitoring capabilities for tech...
For those of you interested in some comprehensive Solr training taught by an expert from Sematext who knows it inside and out, we’re running a super hands-on training workshop in New York City from October 19-20. This two-day workshop will be taught by Sematext engineer — and aut...
This old cat, Apache Tomcat, has been around for ages, but it’s still very much alive!  It’s at version 8, with version 7.x still being maintained, while the new development is happening on version 9.0.  We’ve just added support for Tomcat monitoring to the growing list of SPM integrat...
In vino veritas, right? During a recent team gathering in Kraków, Poland, and after several yummy bottles of țuică, vișinată, żubrówka, diluted with some beer, the truth came out – even though we run Logsene, a log management service that you can think of as hosted ELK Stack, some of ...
For those of you interested in some comprehensive Elasticsearch and ELK Stack (Elasticsearch / Logstash / Kibana) training taught by experts from Sematext who know them inside and out, we’re running a super hands-on training workshop in New York City from October 19-20. This two-day, h...
In this post you’ll learn how to get operational insights (i.e. performance metrics, container events, etc.) from CoreOS and make that super simple with etcd, fleet, and SPM. We’ll use: SPM for Docker to run the monitoring agent as a Docker container and collect all Docker ...
Containers and Docker are all the rage these days. In fact, containers — with Docker as the leading container implementation — have changed how we deploy systems, especially those comprised of microservices. Despite all the buzz, however, Docker and other containers are still relative...
If topics like performance monitoring and processing metrics, log management, and distributed transaction tracing — at scale, no less! — interest you, then you’ll want to check out what Sematext founder Otis Gospodnetić had to say at this week’s DevOps Summit in New York City. Applica...
Docker deployments can be very dynamic with containers being started and stopped, moved around the YARN or Mesos-managed clusters, having very short life spans (the so-called pets) or long uptimes (aka cattle).  Getting insight into the current and historical state of such clusters goe...
Everyone’s infrastructure is growing – whether you run baremetal servers, use IaaS, or use Containers. This just-added SPM functionality, a new view in SPM that we call BEV (aka Birds Eye View) helps you get better visibility into all your servers requiring attention — especially the h...
Sometimes we just need to replay production queries – whether it’s because we want a realistic load test for the new version of a product or because we want to reproduce, in a test environment, a bug that only occurs in production (isn’t it lovely when that happens? E...
Tired of tail -F /your/log/file | egrep -i ‘error|exception|warn’? It’s common for devops to keep an eye out for errors in logs by running tail -F or to manually look for unusual application behavior by looking at logs in their terminal. The problem is that this gets tir...
When digging through logs you might find yourself running the same searches again and again.  To solve this annoyance, Logsene lets you save queries so you can re-execute them quickly without having to retype them: 1) Enter your query and press the “disk” icon next to the search-textbo...
Until software becomes so sophisticated that it becomes truly self-healing without human intervention it will remain important that we humans be notified of any problems with computing systems we run. This is especially true for large or distributed systems where it quickly becomes imp...
In many cases just seeing an unexpected change in log volume is enough to make us want to check out logs to make sure everything is working correctly.  While seeing the general log volume is handy, wouldn’t it be even more useful to see the log volume of errors or exceptions being gene...
CoreOS Linux is the operating system for “Super Massive Deployments”.  We wanted to see how easily we can get CoreOS logs into Elasticsearch / ELK-powered centralized logging service. Here’s how to get your CoreOS logs into ELK in about 5 minutes, give or take.  Here&...
If topics like log analytics and Solr are your thing then we may have a treat for you at the upcoming Lucene / Solr Revolution conference in Austin in October.  Two of Sematext’s engineers and Solr, Elasticsearch and ELK stack experts — Rafal Kuc and Radu Gheorghe — have pr...
Have you read the Top 10 Elasticsearch Metrics to Watch? How about our free eBook – Elasticsearch Monitoring Essentials? If you have, we’re impressed. If not, it’s great bedtime reading. ;) Besides writing bedtime reading material, we also wrote some code last month and added a f...
We just pushed a new release of Logsene to production, including 1-Click Access to Kibana 4! Did you know that Logsene provides a complete ELK Stack? Logsene’s indexing and search API is compatible with the Elasticsearch API.  That’s why it is very easy to use Logsene – you...
Elasticsearch is booming.  Together with Logstash, a tool for collecting and processing logs, and Kibana, a tool for searching and visualizing data in Elasticsearch (aka the “ELK stack”), adoption of Elasticsearch continues to grow by leaps and bounds. In this detailed (and free!) book...
For those of you attending next week’s DevOps Summit event in New York City (part of the larger Cloud Computing Expo) with an interest in topics like performance monitoring and processing metrics, log management, and distributed transaction tracing — at scale, no less! — then Sematext ...
The release of Kibana 4.x has had an impact on monitoring and other related activities.  In this post we’re going to get specific and show you how to add Node.js monitoring to the Kibana 4 server app.  Why Node.js?  Because Kibana 4 now comes with a little Node.js server app that...
Back by popular demand!  Sematext engineers Radu Gheorghe and Rafal Kuc are returning to Berlin Buzzwords next Tuesday, June 2, with the second installment of their “Side by Side with Elasticsearch and Solr” talk.  (You can check out Part 1 here.) Elasticsearch and Solr Performance and...
Here at Sematext we use Java (8!) and rely on centralized logging a lot. We like them so much that we regularly share our logging experience with everyone and help others with logging, especially, ELK stack. Centralized logging plays nice with Java (and anything else that can write pre...
By popular demand…we’ve just added some new goodness to SPM with the SPM REST API. This new API lets you: create SPM Apps for monitoring (e.g. generate a new SPM App + its token during deployment; list all available metrics and charts for a specific App; list all alerts defined for so...
Elasticsearch is booming. Together with Logstash, a tool for collecting and processing logs, and Kibana, a tool for searching and visualizing data in Elasticsearch (aka the “ELK” stack), adoption of Elasticsearch continues to grow by leaps and bounds. When it comes to actually using ...