(eBook) Practical Redis: first few chapters released!

I am happy to announce that the first few chapters of Practical Redis are now available

About Practical Redis

cover5

 

It is a hands-on, code-driven guide to Redis where each chapter is based on an application (simple to medium complexity) which demonstrates the usage of Redis and its capabilities (data structures, commands etc.).

The applications in the book are based on Java and Golang. The Java based Redis client libraries which have been used in this book include JedisRedisson and Lettuce while go-redis is used as the client library for Golang. Docker and Docker Compose are used to deploy the application stack (along with Redis)

 

Here is a quick outline of the book contents

  • Hello Redis – quick tour of Redis capabilities including its versatile data structures
  • Redis: the basic data structures – intro to core Redis data structures with a Java based news sharing app using the Jedisclient
  • Extending Redis with Redis Modules – learn about the basics of Redis Modules and make use of ReBloom in a Go based recommendation service
  • Tweet analysis service – keep track of relevant tweets using a Java and Go based application which ingests tweets, consumes/processes them and produces queryable info in real time using reliable queues (Redis LISTs), SETs and HASHes

Coming soon …

The below chapters are work in progress on and will be made available in subsequent releases of this book

  • Pipelines and Transactions in Redis
  • Scalable chat application – Use Redis PubSub and Websocket to create a chat service
  • Stream processing with Redis – watch Redis Streams in action (new data structure in Redis 5.0)
  • Redis based Tracking service – thanks to combination of the Geo data type and Lua scripting capability in Redis
  • Redis for distributed computing – explore interesting use cases made possible by Redisson
  • Data partitioning in Redis – practical examples highlighting data sharding strategies in Redis
  • Redis high availability

Go check it out & stay tuned for more updates

Cheers!

Advertisements
Posted in Distributed systems, go, nosql, redis | Tagged , , , , , , , , | Leave a comment

Redis geo.lua example using Go

I stumbled upon geo.lua which seemed to be an interesting library

It’s described as – “… a Lua library containing miscellaneous geospatial helper routines for use with Redis

Here is an example of using it with the Go Redis client (go-redis). This is what it does in a nutshell

To run, refer README – its super simple

Cheers!

Posted in go, nosql, redis | Tagged , , , , , , | Leave a comment

certificate error with Go HTTP client in alpine Docker

Using the Go HTTP client from a alpine docker image will result in this error – x509: failed to load system roots and no roots provided

Solution: alpine is a minimal image, hence CA certificates are required. You add that in the Dockerfile as per below example

FROM alpine 
RUN apk add --no-cache ca-certificates 
COPY my-app .
CMD ["./my-app"]

Got this hint from the golang alpine Dockerfile
Cheers!
Posted in go | Tagged , , , | Leave a comment

etcd Watch example using Go client

etcd is a distributed, highly available key-value store which serves as the persistent back end for Kubernetes

etcd

Here is an example to demonstrate its Watch feature. For details, check out the README (and the sample code) on Github

It uses

Summary

Cheers!

Posted in Distributed systems | Tagged , , , | Leave a comment

NATS on Kubernetes example

In a previous blog, you saw an example of NATS along with producer and consumer – based on Docker Compose. This post uses the same app, but runs on Kubernetes

nats-k8s.jpg

For how to run, follow the README on Github

High level overview

  • uses the NATS Kubernetes operator
    • it seeds a CRD …
    • … using which we can spawn a NATS cluster on Kubernetes (with a single command!)
    • A Service object is created automatically – uses ClusterIP by default
nats-cluster-service.JPG

NATS Kubernetes Service object

  • The environment variable corresponding to the Serviceis used in the application logic to locate NATS server. Note its usage here

Cheers!

nats-logo.png

Posted in Distributed systems, Kubernetes, nats | Tagged , , , , , , , , | Leave a comment

Redis tech tip #1

In order to delete members from your Redis Geo-spatial collection, you can use ZREM (underneath the hood, the data is stored in a sorted set)

Add a few (Indian) cities

GEOADD IndianCities 28.6108127 77.2060241 NewDelhi

​GEOADD IndianCities 12.9715987 77.5945627 Bangalore

GEOADD IndianCities 13.0826802 80.2707184 Chennai

The weather in Chennai humid – not my kind.. let’s remove it

ZREM IndianCities Chennai

You can also delete the entire collection using DEL IndianCities (usual stuff…)

Cheers!

Posted in nosql, redis | Tagged , , , | Leave a comment

NATS & Kafka: random notes

This is not NATS vs Kafka by any means – just jotting down some info. NATS recently joined CNCF (which host projects like Kubernetes, Prometheus etc. – look at the dominance of Golang here!) and that’s when it caught my attention

I have been taken aback by its simplicity and performance (yet to test drive it fully – but the numbers are out there)

  • Kafka runs on JVM (Scala to be specific), NATS is written in Go
  • Protocol – Kafka is binary over TCP as opposed to NATS being simple text (also over TCP)
  • Messaging patterns – Both support pub-sub and queues, but NATS supports request-reply as well (sync and async)
  • For (horizontally) scalable processing,
    • NATS has a concept of queue (with a unique name of course) and all the subscribers hooked on same queue end up being a part of the same queue group. Only one of the (potentially multiple) subscribers gets the message. Multiple such queue groups would also receive the same set of messages. This makes it a hybrid pub-sub (one-to-many) and queue (point-to-point)
    • Same thing is supported in Kafka via consumer groups​ which can pull data from one or more topics
  • Stream processing – NATS does not support stream processing as a first class feature like Kafka does with Kafka Streams
  • Kafka clients use a poll-based technique in order to extract messages as opposed to NATS where the server itself routes messages to clients (maintains an interest-graph internally)
  • NATS can act pretty sensitive in the sense that it has the ability to cut off consumers who are not keeping pace with the rate of production as well as clients who don’t respond to heartbeat requests.
    • The consumer liveness check is executed by Kafka as well. From what I recall, this is done/initiated from the client itself & there are complex situations that can arise due to this (e.g. when you’re in a message processing loop and don’t poll ). There are a bunch of configuration parameter/knobs to tune this behavior (on client side)
  • Delivery semantic – NATS supports at-most once (and at-least-once with NATS streaming) as opposed to Kafka which also supports exactly-once (tough!)
  • NATS doesn’t seem to have a notion of partitioning/sharding messages like Kafka does
  • No external dependency in case of NATS. Kafka requires Zookeeper

… work in progress…

 

Posted in Distributed systems, Kafka, messaging | Tagged , , , | Leave a comment

NATS – quick start with Docker

Here is a hello-world example (on Github) for getting started with natsio – a distributed messaging server written in Go

It consists of the following components which are managed via Docker Compose

Cheers!

Posted in Distributed systems, messaging | Tagged , , , , | 1 Comment

Session @ Oracle Code San Francisco 2017

Here are the details for my talk @ Oracle Code (a track at JavaOne 2017) – Streaming Solutions for Real time problems (Stream Processing solutions using Apache Kafka, Kafka Streams and Redis)

Code (Github) – https://github.com/abhirockzz/accs-ehcs-stream-processing

Video

Slides

Cheers!

Posted in Distributed systems, Kafka | Tagged , , , , , , , , , | Leave a comment

Debezium test drive

Debezium is an open source, distributed change data capture system built on top of Apache Kafka. I tried it out and the project is available on Github

Setup

Details are in the README. It uses the Debezium tutorial as a background but the setup is very simple since it uses Docker Compose – one command (docker-compose up --build is all it takes to get going !

Components

  • MySQL – the database
  • Debezium – the CDC platform which tails MySQL binlog and pushes its logs to Kafka using its MySQL Kafka Connect connector
  • Consumer – its a Java EE application which consumes the (DB change) events from Kafka (topic), derserializes and parses it out and logs it

results.jpg

Cheers!

 

Posted in Distributed systems, Kafka | Tagged , , , , , , | 1 Comment