Welcome!

Enterprise DevOps, Log Management and Analytics

Sematext Blog

Subscribe to Sematext Blog: eMailAlertsEmail Alerts
Get Sematext Blog via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Latest Blogs from Sematext Blog
Use-case: you’re using AWS VPC and want visibility over the connection to your VPC. Which IPs are allowed or denied connections to certain ports, how much traffic goes through each connection and so on. Solution: send AWS VPC logs (one type of CloudWatch logs) to a Logsene applic...
If you are running apps on top of JVM and want to be able to profile them in production, on-demand, without affecting your app’s performance and users, read on!  Screenshots, features, and other juicy stuff is further down.Do you run any apps on the JVM?  How do you find bottlene...
Last week, we upgraded Logsene to Elasticsearch v2.2.0 including the required upgrade to Kibana 4.3.1.  This means you can benefit from Elasticsearch 2.2.0, Kibana 4.3.1 and the updated Logsene UI using Elasticsearch 2.x features. In addition, we migrated all existing data from Elastic...
Last week, we upgraded Logsene to Elasticsearch v2.2.0 including the required upgrade to Kibana 4.3.1.  This means you can benefit from Elasticsearch 2.2.0, Kibana 4.3.1 and the updated Logsene UI using Elasticsearch 2.x features. In addition, we migrated all existing data from Elastic...
Fresh from Germany’s largest Node.js Meetup, hosted by Wikimedia in Berlin, is the latest presentation from Sematext DevOps Evangelist Stefan Thies — “Top Node.js Metrics To Watch”. The event was shared with the Node.js Meetup in London via video-live stream, the full recording i...
Fresh from Germany’s largest Node.js Meetup, hosted by Wikimedia in Berlin, is the latest presentation from Sematext DevOps Evangelist Stefan Thies — “Top Node.js Metrics To Watch”. The event was shared with the Node.js Meetup in London via video-live stream, the full recording i...
Heroku is a cloud platform based on a managed container system, with integrated data services and a powerful ecosystem for deploying and running modern apps.  In this post we’ll show how you can ship logs from Heroku to Logsene, where you can then search your logs, get alerts based on ...
What is the easiest way to parse, ship and analyze my web server logs? You should know that I’m a Node.js fan boy and not very thrilled with the idea of running a heavy process like Logstash on my low memory server, hosting my private Ghost Blog. I looked into Filebeat, a very light-we...
Sematext team is highly distributed. We are ex-Skype users who recently switched to Slack for team collaboration. We’ve been happy with Slack features and especially integrations for watching our Github repositories, Jenkins, or receiving SPM or Logsene Alerts from our production serve...
3 Elasticsearch Classes in London Elasticsearch for Developers ……. April 4-5 Elasticsearch for Logging ……… April 6 Elasticsearch Operations …….  April 6 All classes cover Elasticsearch 2.x Hands-on — lab exercises follow each class section Early bird pricing until February 29 Add...
April 4 & 5 — Covers Solr 5.x Hands-on — lab exercises follow each class section Early bird pricing until February 29 Add a second seat for 50% off Sematext is running a 2-day, very comprehensive, hands-on workshop in London on April 4 & 5 for Developers and DevOps who want to ...
There are a lot of sources of logs these days. Some may come from mobile devices, some from your Linux servers used to host data, while other can be related to your Docker containers. They are all supported by Logsene. What’s more, you can also ship logs from your Microsoft Windo...
The official Solr Image on Docker Hub was released just a few weeks ago and already has 16K pulls. Why not more? Well, there are more than 200 different Solr images on Docker Hub — probably because no official Image was available! A rapidly growing number of organizations are using So...
Running Elasticsearch clusters on Docker? Thinking about it?  If “yes” then we’ve got a presentation for you that digs deep into the details. (Note: we’ve also got a related blog post about monitoring the official Elasticsearch image on Docker that you might find useful) Coming to you ...
One of the nice things about our log management and analytics solution Logsene is that you can talk to it using various log shippers.  You can use Logstash, or you can use syslog protocol capable tools like rsyslog, or you can just push your logs using the Elasticsearch API just like y...
Great news for for those of us who use PagerDuty and manage — or are considering managing — logs with Logsene: PagerDuty and Logsene are now integrated! This integration is a huge time- and aggravation-saver for DevOps professionals who wouldn’t mind dramatically reducing t...
New Year, New Feature in SPM!  We are happy to announce the immediate availability of NetMaps in SPM!  Check out why they are useful or watch the short video below. Ever wondered how different components of distributed apps are actually connected over the network? When it comes to trou...
This recipe shows how to send CloudTrail logs (which are .gz logs that AWS puts in a certain S3 bucket) to a Logsene application, but should apply to any kinds of logs that you put into S3. We’ll use AWS Lambda for this, but you don’t have to write the code. We’ve got...
Another year is behind us, and it’s been another good year for us at Sematext.  Here are the highlights in the chronological order.  If you prefer looking non-chronological overview, look further below. January We started the year by doing a ton of publishing on the blog – about ...
Sematext has combined the power of SPM and Logsene in a single pane of glass – a unified view into all the key bits of operational intelligence every DevOps engineer needs: server and application performance metrics, logs, events, anomalies, alerts, ChatOps integrations, etc. In other...
If you’ve found your way to this post then chances are high that you’re having second thoughts about diving into a New Relic APM subscription.  You’re not alone.  In fact, we hear from many fellow DevOps engineers looking at performance monitoring solutions who check out New Relic APM ...
For many of us in the DevOps field, MongoDB is a critical part of our IT stack.  With yesterday’s acquisition of WiredTiger, MongoDB is further establishing itself as the NoSQL DB built to support massive data processing and storage.  It would be an understatement to say that Mon...
Sematext has just been recognized by Docker as an Ecosystem Technology Partner (ETP) for logging. This designation indicates that Logsene has contributed to the logging driver and is available to users and organizations that seek solutions to capture logging data for monitoring their D...
Grafana is an open-source alternative to Kibana.  Grafana is best known as a visualization / dashboarding tool focused on graphing metrics from various data sources, such as InfluxDB. Even though Grafana started its life as a Kibana fork, it didn’t originally support using Elasticsearc...
Running Elasticsearch clusters on Docker? Thinking about it?  If “yes” then we’ve got a presentation for you that digs deep into the details. (Note: we’ve also got a related blog post about monitoring the official Elasticsearch image on Docker that you might find useful) Fresh from Dev...
We’ve been hard at work on our centralized logging SaaS / On-Premises solution – Logsene – and we’re confident the logging fans among us will enjoy the new Live Tail functionality. Live Tail Benefits Logsene Live Tail has several important benefits, including: shows logs in...
If you run Elasticsearch, Solr, or any datastore you connect to via JDBC, you’ll like what we’ve just added to SPM.  We call it Database Operations and in SPM you can find it in the new Database report: If you didn’t watch the video, here’s what Database Operations gives you: Top 5 ope...
Half of the world, Sematext included, seems to be using Kafka. Kafka is the spinal cord that connects various components in SPM, Site Search Analytics, and Logsene.  If Kafka breaks, we’re in trouble (but we have anomaly detection all over the place to catch issues early).  In ma...
The official Elasticsearch Image on Docker Hub has already generated more than 1.6 million pulls. It is probably the easiest way to get a development setup — which includes Elasticsearch — to the application stack. The reason for this crazy number? A rapidly growing number ...
There are many ways to look at Elasticsearch, but here at Sematext we’re pretty confident that you haven’t seen anything like this flowchart to demonstrate how it works: Download a copy and print your own Elasticsearch poster! If you’re looking for something unique to show off your Ela...
In this presentation from Lucene/Solr Revolution 2015, Sematext engineers — and Solr and centralized logging experts — Radu Gheorghe and Rafal Kuć talk about searching and analyzing time-based data at scale. Documents ranging from blog posts and social media to application ...
Fresh from the Velocity NYC conference is the latest presentation from Sematext engineers Rafal Kuć and Radu Gheorghe — “From zero to production hero: Log Analysis with Elasticsearch.” The talk goes through the basics of centralizing logs in Elasticsearch and all the ...
Docker Logging has been a very popular topic of late in our internal and external discussions.  So much so that we decided to hold webinars on the topic (and Docker Monitoring as well) and now we’re making them available to everyone. The webinars were presented by Sematext’s DevOps Eva...
The recent Docker Monitoring webinar is ready for consuming!  Our DevOps Evangelist, Stefan Thies, took attendees on a tour of Docker monitoring basics, including a number of different Docker monitoring options and their pros and cons, solutions for Docker monitoring, and a brief Q&...
This recipe is similar to the previous rsyslog + Redis + Logstash one, except that we’ll use Kafka as a central buffer and connecting point instead of Redis. You’ll have more of the same advantages: rsyslog is light and crazy-fast, including when you want it to tail files a...
We’ve been hard at work on our centralized logging SaaS / On-Premises solution – Logsene – and we’re confident the logging fans among us will enjoy the new Live Tail functionality. Live Tail Benefits Logsene Live Tail has several important benefits, including: shows logs in...
More than two years ago we posted a recipe on how to centralize syslog in Elasticsearch in order to search and analyze them with Kibana, all by using only rsyslog. This works well, because rsyslog is fast and light, as we shown in later posts and recent presentations. This recipe is ab...
The results for the log shipping formats poll are in.  Thanks to everyone who took the time to vote! The distribution pie chart is below, but we can summarize it for you here: JSON won pretty handily with 31.7% of votes, which was not totally unexpected. If anything, we expected to see...
Many Solr users need to handle multi-tenant data. There are different techniques that deal with this situation: some good, some not-so-good. Using routing to handle such data is one of the solutions, and it allows one to efficiently divide the clients and put them into dedicated shards...
OK, so you want to hook up rsyslog with Logstash. If you don’t remember why you want that, let me give you a few hints: Logstash can do lots of things, it’s easy to set up but tends to be too heavy to put on every server you have Redis already installed so you can […]