
Microservices with Go
Description
Alles über E-Books | Antworten auf Fragen rund um E-Books, Kopierschutz und Dateiformate finden Sie in unserem Info- & Hilfebereich.
- Discover industry best practices and gain insights into Go microservice development tools, patterns, and solutions
- Work with hands-on Golang microservice examples in each chapter
- Purchase of the print or Kindle book includes a free PDF eBook
Book DescriptionMicroservices with Go, Second Edition explains the key benefits and common issues faced by developers when working with microservices, helping you understand the problems microservice architecture solves, the issues it introduces, and how to tackle them. The author distils his 18+ years of experience in building scalable and reliable infrastructure to help you grasp the importance of using the right principles and standards to achieve all that microservice architecture has to offer. You'll see why Go is a popular choice for microservice development, as well as navigate its foundational aspects, including service scaffolding, discovery, data serialization, communication, deployment, and testing. After covering development, you'll move to maintenance and reliability. This second edition is fully updated with newly added topics, including security and compliance, distributed system challenges, and performance monitoring. The final section focuses on advanced concepts, such as system reliability, observability, maintainability, and scalability. Through best practices and practical examples, you'll learn how to apply key ideas to existing applications using previously scaffolded services. By the end of this book, you'll have gained hands-on experience in developing scalable, reliable, and high-performance microservices with Go.What you will learn - Leverage event-driven architecture and messaging patterns for efficient microservice communication
- Automate testing, CI/CD pipelines, and deployment strategies for Go microservices
- Establish secure communication between microservices
- Optimize microservice performance by monitoring, profiling, and debugging techniques
- Get hands-on experience in cloud-native observability and deployment tools
- Explore solutions to advanced distributed system scenarios, such as leader election
Who this book is forThis book is for all types of developers, from individuals interested in learning how to write microservices in Go to seasoned professionals who want to master the art of writing scalable and reliable microservice-based systems. A basic understanding of Go will come in handy.
All prices
More details
Other editions
Previous edition

Content
- Cover
- Title Page
- Contributors
- Preface
- Free benefits with your book
- Part 1: Introduction
- Chapter 1: Introduction to Microservices
- Your purchase includes a free PDF copy + exclusive extras
- What is a microservice?
- Motivation to use microservices
- Pros and cons of microservices
- Benefits of microservices
- Common issues of microservices
- When to use microservice architecture
- The role of Go in microservice development
- Summary
- Further reading
- Part 2: Foundation
- Chapter 2: Scaffolding a Go Microservice
- Technical requirements
- Go basics
- Core principles
- Writing idiomatic Go code
- Naming
- Comments
- Errors
- Interfaces
- Tests
- Context
- Project structure
- Private packages
- Public packages
- Executable packages
- Other commonly used directories
- Common files
- Best practices
- Scaffolding an example application
- Movie application
- Movie metadata
- Ratings
- Should we split the application?
- Application code structure
- Movie metadata service
- Model
- Repository
- Controller
- Handler
- Main file
- Rating service
- Model
- Repository
- Controller
- Handler
- Main
- Movie service
- Model
- Gateways
- Controller
- Handler
- Main file
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 3: Service Discovery
- Technical requirements
- Service discovery overview
- Registry
- Service discovery models
- Client-side service discovery
- Server-side service discovery
- Service health monitoring
- Service discovery solutions
- HashiCorp Consul
- Kubernetes
- etcd
- Apache ZooKeeper
- Adopting service discovery
- Preparing the application
- Implementing the discovery logic
- In-memory implementation
- Consul-based implementation
- Using the discovery logic
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 4: Serialization
- Technical requirements
- The basics of serialization
- Popular serialization formats
- XML
- YAML
- Apache Thrift
- Apache Avro
- Protocol Buffers
- Using Protocol Buffers
- Serialization best practices
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 5: Synchronous Communication
- Technical requirements
- Introduction to synchronous communication
- Go RPC frameworks and libraries
- Apache Thrift
- gRPC
- Defining a service API using Protocol Buffers
- Implementing gateways and clients
- Metadata service
- Rating service
- Movie service
- Synchronous communication best practices
- Perform excessive request validation and return correct error codes
- Ensure idempotency
- Summary
- Further reading
- Chapter 6: Asynchronous Communication
- Technical requirements
- Asynchronous communication basics
- Benefits and challenges of asynchronous communication
- Techniques and patterns of asynchronous communication
- Message broker
- The publisher-subscriber model
- Using Apache Kafka for messaging
- Apache Kafka basics
- Adopting Kafka for our microservices
- Asynchronous communication best practices
- Versioning
- Leverage partitioning
- Use explicit message acknowledgment whenever necessary
- Use a separate topic for unprocessed messages
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 7: Storing Service Data
- Technical requirements
- Introduction to databases
- Common database features
- Common database types
- Using MySQL to store our service data
- Implementing data caching
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 8: Setting Up Service Deployments
- Technical requirements
- Preparing application code for deployments
- Deployment basics
- Application configuration
- Service deployment solutions
- Docker Swarm
- HashiCorp Nomad
- Kubernetes
- Deploying via Kubernetes
- Introduction to the Kubernetes data model
- Setting up our microservices for Kubernetes deployments
- Deployment best practices
- Automated rollbacks
- Canary deployments
- Continuous deployment
- Use feature flags to decouple deployments from feature releases
- Summary
- Further reading
- Chapter 9: Unit and Integration Testing
- Technical requirements
- Go testing overview
- Subtests
- Skipping
- Unit tests
- Mocking
- Implementing unit tests
- Integration tests
- Testing best practices
- Using helpful messages
- Avoiding the use of Fatal in your logs
- Comparing structures using a cmp library
- Detecting and fixing flaky tests
- Detecting race conditions
- Tracking and maintaining high code coverage
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 10: Security and Compliance
- Technical requirements
- Security basics
- Key areas of software security
- Authentication and access control
- Secure service communication
- Implementing secure service communication with TLS
- Additional improvements
- Implementing authentication and access control with JWT
- The basics of JWT
- Implementing JWT issuance and validation in microservices
- Security analysis of Go services
- Using gosec to automate security analysis
- Security best practices
- Secure secret management
- Use automated vulnerability scanning
- Perform periodic threat modeling exercises
- Compliance basics
- Summary
- Further reading
- Get this book's PDF version and more
- Part 3: Maintenance
- Chapter 11: Reliability Overview
- Technical requirements
- Reliability basics
- Achieving reliability through automation
- Communication error handling
- Implementing request retries
- Deadlines and timeouts
- Fallbacks
- Rate limiting
- Graceful shutdown
- Achieving reliability through development processes and culture
- On-call process
- Incident management
- Reliability drills
- Disaster recovery plans
- Set and track the reliability objectives of your services
- Summary
- Further reading
- Chapter 12: Collecting Service Telemetry Data
- Technical requirements
- Telemetry overview
- Collecting service logs
- Choosing a logging library
- Using logging features
- Storing microservice logs
- Elasticsearch
- OpenSearch
- OpenTelemetry Collector
- Logging best practices
- Avoiding using interpolated strings
- Standardizing the format of your log messages
- Periodically reviewing your log data
- Setting up appropriate log retention
- Identifying the message source in logs
- Collecting service metrics
- Storing metrics
- Prometheus
- Graphite
- OpenTelemetry Collector
- Popular Go metrics libraries
- Emitting service metrics
- Metrics best practices
- Keeping tag cardinality in mind
- Standardizing metric and tag names
- Setting the appropriate retention
- Collecting service traces
- Tracing tools
- Collecting tracing data with the OpenTelemetry SDK
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 13: Setting Up Service Alerting
- Technical requirements
- Alerting basics
- Alerting use cases
- Introduction to Prometheus
- Setting up Prometheus alerting for our microservices
- Alerting best practices
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 14: Performance Monitoring
- Technical requirements
- Creating dashboards to visualize service telemetry data
- Introduction to dashboards
- Creating performance dashboards using the Grafana tool
- Adding service-level metrics to dashboards
- Profiling Go services
- Profiling CPU usage using the pprof tool
- Profiling heap memory usage using the pprof tool
- Summary
- Further reading
- Get this book's PDF version and more
- Part 4: Advanced Topics
- Chapter 15: Implementing Distributed System Scenarios
- Technical requirements
- Introduction to distributed system problems
- Consensus in distributed systems
- Paxos
- Raft
- Distributed locking and leader election
- Distributed system tools
- Apache Zookeeper
- etcd
- HashiCorp Consul
- Implementing distributed locking with HashiCorp Consul
- Distributed system best practices
- Build systems with minimal necessary coordination between components
- Avoid distributed transactions whenever possible and find the right level of consistency for your data
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 16: Advanced Topics
- Technical requirements
- Static analysis of Go service code
- Implementing data validation
- Implement streaming APIs
- Frameworks
- Storing microservice ownership data
- Summary
- Further reading
- Get this book's PDF version and more
- Chapter 17: Unlock Your Exclusive Benefits
- Unlock this book's free benefits in 3 easy steps
- Other Books You May Enjoy
- Index
1
Introduction to Microservices
In this chapter, you will be introduced to microservices and the motivation behind them. You will understand the key benefits and common issues of the microservice architecture model and learn when to use it, as well as learning about some microservice development best practices. This knowledge will help you establish a solid foundation for reading the next chapters and give you some ideas on what challenges you may face with microservices in the future.
In this chapter, we will cover the following topics:
- What is a microservice?
- Motivation to use microservices
- Pros and cons of microservices
- When to use microservice architecture
- The role of Go in microservice development
Your purchase includes a free PDF copy + exclusive extras
Your purchase includes a DRM-free PDF copy of this book, 7-day trial to the Packt+ library (no credit card required), and additional exclusive extras. See the Free benefits with your book section in the Preface to unlock them instantly and maximize your learning.
What is a microservice?
Companies worldwide have used the microservice architecture model so widely that it has almost become a default way of software development. There are many companies having tens, hundreds, and even thousands of microservices at their disposal.
So, what exactly is the microservice model?
The microservice architecture model is organizing an application as a collection of services, called microservices, each of which is further responsible for a certain part of application logic, usually defined by a particular business capability.
As an example, consider an online marketplace application. The application may have multiple features, including search, a shopping cart, payments, order history, and much more. Each feature can be so different and complex on its own that it could be developed and maintained by a separate team - like search and payments in our example. In the microservice architecture model, each component would be an independent service playing its own role in the system.
Organizing each part of the application as a separate service is not necessarily a requirement. As with any architecture model or any aspect of software development, engineers need to be careful with choosing a particular approach or solution - doing an initial analysis and understanding the solution under the given conditions.
Before we proceed to the key benefits and downsides of microservices, let's see what challenges you could face when the application is not separated into multiple services.
Motivation to use microservices
In order to understand the motivation behind using the microservice architecture, it is very important to see the opposite approach - when the application is built and executed as a single program. Such applications are called monolithic applications or monoliths.
Monolithic architecture is, in most ways, the simplest model to implement since it does not involve splitting the application into multiple parts that need to coordinate with each other. This can provide you with major advantages in many cases, such as the following:
- Application logic is still loosely defined: It is very common that parts of the application or the entire system go through major structural or logical changes, especially at the very early stages of development. This might be caused by a sudden change of requirements, priorities, changes in the business model, or a different approach to development. In such cases, the ability to make quick changes to the application structure can be critical not only to the development process but also to the entire company.
- Narrow scope of the application: Certain applications can have a limited scope (for example, a photo upload service) or have their features integrated tightly (payment processing and invoice generation). Such applications might benefit from being monoliths: it is much easier to perform end-to-end testing and development without extra boundaries between the parts of the application.
- Performance constraints: There are types of applications (for example, trading systems) that can be very sensitive to the latency of each operation. Time-sensitive parts of such applications could benefit from monolithic architecture because there wouldn't be any API calls between the components.
In all of the preceding cases, monolithic architecture would be a better fit for the application. However, at some point, services get too big to remain monolithic. Developers start experiencing the following issues:
- Large application size and slow deployments: At a certain point, an application can become so big that it can take minutes or even hours to build, start, or deploy.
- Inability to deploy a particular part of the application independently: Not being able to replace a part of a large application can easily become a bottleneck, slowing down the development and release process.
- Higher blast radius: If there is a bug in a certain function or library widely used across the application code, it is going to affect all parts of the system at once, potentially causing major issues.
- Inability to scale components separately: If some part of a monolithic application requires more resources (CPU, RAM, or disk space), the entire application would need to be scaled - this could be very expensive.
- Vertical scalability bottleneck: The more logic the application has, the more resources it needs in order to run. At a certain point, it can get hard or impossible to scale the application up even further, given the possible limits on CPU and RAM.
- Interference: Certain parts of the application can perform heavy operations (for example, CPU-intensive data processing), reducing the performance of the rest of the system.
- Higher security risks: A possible security issue in the application may result in unauthorized access to all components at once.
- In addition to the possible issues we just described, different components may have different requirements, such as the following:
- Resources and hardware requirements: Certain components are more CPU-intensive or memory-intensive and may perform I/O operations at a higher rate. Separating such components may reduce the load on the entire system, increasing system availability and reducing latency.
- Deployment cadence: Some parts of the system mostly remain unchanged while others require multiple deployments per day.
- Deployment monitoring and automated testing: Certain components may require stricter checks and monitoring and can be subject to slower deployments due to multi-step rollouts.
- Technologies or programming languages: It is not uncommon that different parts of the system can be written in different programming languages or use fundamentally different technologies, libraries, and frameworks.
- Change management process: Some components may be subject to a stricter code review, deployment, or data management processes, as well as additional requirements.
- Security: Components may have different security requirements and may require additional isolation from the rest of the application for security reasons.
- Compliance: Some parts of the system may be subject to stricter compliance requirements. For example, handling personally identifiable information (PII) for users from a certain region can put stricter requirements on the entire system. Logical separation of such components helps to reduce the scope of work required to keep the system compliant.
With all the preceding issues described, we can see that, at a certain point, monolithic applications can become too big for a one-size-fits-all model. As the application grows, certain parts of it may start becoming independent and have different requirements, benefiting from a logical separation from the rest of the application.
In the next section, we are going to see how splitting the application into microservices can solve the aforementioned problems and which aspects of it you should be careful with.
Pros and cons of microservices
In order to understand how to get the best results from using microservices and which issues to be aware of, let's review the pros and cons of the microservice model.
Benefits of microservices
As previously described, different application components may have fundamentally different requirements and, at certain points, diverge so much that it would be beneficial to separate them. In this case, microservice architecture provides a clear solution by decoupling the parts of the system.
Microservices provide the following benefits to developers:
- Faster compilation and build time:...
System requirements
File format: ePUB
Copy protection: Adobe-DRM (Digital Rights Management)
System requirements:
- Computer (Windows; MacOS X; Linux): Install the free reader Adobe Digital Editions prior to download (see eBook Help).
- Tablet/smartphone (Android; iOS): Install the free app Adobe Digital Editions or the app PocketBook before downloading (see eBook Help).
- E-reader: Bookeen, Kobo, Pocketbook, Sony, Tolino and many more (not Kindle).
The file format ePub works well for novels and non-fiction books – i.e., „flowing” text without complex layout. On an e-reader or smartphone, line and page breaks automatically adjust to fit the small displays.
This eBook uses Adobe-DRM, a „hard” copy protection. If the necessary requirements are not met, unfortunately you will not be able to open the eBook. You will therefore need to prepare your reading hardware before downloading.
Please note: We strongly recommend that you authorise using your personal Adobe ID after installation of any reading software.
For more information, see our ebook Help page.
File format: ePUB
Copy protection: without DRM (Digital Rights Management)
System requirements:
- Computer (Windows; MacOS X; Linux): Use a reader that can handle the file format ePUB, such as Adobe Digital Editions or FBReader – both free (see eBook Help).
- Tablet/Smartphone (Android; iOS): Install the free app Adobe Digital Editions or the app PocketBook (see eBook Help).
- E-reader: Bookeen, Kobo, Pocketbook, Sony, Tolino and many more (not Kindle).
The file format ePUB works well for novels and non-fiction books – i.e., 'flowing' text without complex layout. On an e-reader or smartphone, line and page breaks automatically adjust to fit the small displays.
This eBook does not use copy protection or Digital Rights Management
For more information, see our eBook Help page.