Welcome!

Enterprise DevOps, Log Management and Analytics

Sematext Blog

Subscribe to Sematext Blog: eMailAlertsEmail Alerts
Get Sematext Blog via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Related Topics: MySQL Journal

Blog Feed Post

Taming SwarmZilla: 150k Containers in 3K+ Docker Swarm Nodes

SwarmZilla/swarm3k by Docker Captain Chanwit Kaewkasi is a unique community project/event aimed at launching a Docker Swarm cluster with 3000+ community-sponsored nodes. The previous project – Swarm2k – successfully demonstrated a 2000+ node Swarm cluster with only 3 Swarm managers running a workload with 95,000 tasks/containers on worker nodes.

Swarm3k goal is to run more than 3,000 nodes with a very large subnet – with 4096 IP addresses and several workloads, such as a 20 node MySQL cluster and 2,980 WordPress tasks on top of it.

Schedule:

  • Test run: October 19, 2016, 3 PM UTC – smaller test run – creating nodes, deploying services, etc. 
  • Real run: October 28, 2016, 3 PM UTC

Joining:

You can join with your own Docker containers. See swarmzilla/swarm3k for instructions.  Got questions?  We’re in swarmzilla Gitter chat.

Observing / Monitoring:

Sematext is the official Docker Swarm3k monitoring partner – see Live Swarm3k Dashboard at http://sematext.com/swarm3k. Compared to the setup for Swarm2k we will enrich the monitoring setup with host metrics, container metrics and logging of Docker events and task errors.  All these bits of operational data are collected by Sematext Docker Agent.

Finally, we tweaked Sematext Docker Agent for the Swarm3k to deal with its special requirements. In addition, we decided to sponsor the resources for the monitoring including a dedicated extension of our infrastructure. As the result we get:

Swarm cluster overview including task monitoring (updated screeshot, Oct. 28, 2016)

If you are interested in this topic, stay tuned and follow Chanwit Kaewkasi and @sematext and check out the public Docker Swarm monitoring dashboard.  If you need Docker monitoring for your own Docker deployments check out SPM for Docker, the Sematext Docker Agent, and Logsene.

Read the original blog entry...

More Stories By Sematext Blog

Sematext is a globally distributed organization that builds innovative Cloud and On Premises solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), and search analytics (SSA). We also provide Search and Big Data consulting services and offer 24/7 production support for Solr and Elasticsearch.