Motivation

For my independent project, I have chosen to create a Microservices app. I have never developed an entire app in microservices architecture, so it will be challenging. If you don't know what a microservices architecture is, here is a link to my mini research that sums up the most important parts.

The app will be part of my personal project of fictional NIC Athletics. I have created a database data schema that might help you picture the app's scope. However, I intend this to be more of a POC than a project I would put into my resume.

Technology used: Kotlin, Micronaut, JavaScript, NATS, PostreSQL and more.

Study plan

I plan to create these three services.

Goals	Week
Set up environment	0
Create first services	4
Integrate registration service	5
Integrate team service	6
Implement messaging stream system	7

Week 0

This week I focus on setting up an environment and creating three initial services, including technologies such as Skaffold, Docker, and Ingress.

Spent time: ~12 hours

Challenges

• Creating Micronaut services and integrating them with Skaffold and Docker
• Designing the structure of the services
• I tried to implement Ktor as well. It took too much time to fix bugs.

Progress

Micronaut has a pretty well-documented process of how to implement Skaffold. On the other side, implementing Docker went remarkably worse. I use Grandle, and in the documentation, Maven solution was shown. It took me a couple of hours to find the right solution. After implementing both, I could implement Ingress, which works like a load balancer (It is not!). Anytime a client requests the server, Ingress points to the appropriate service based on the URL. I also implemented auto-deployment to Google Cloud by Skaffold. You can see that in skaffold-gcr.yml. However, the project will be developed locally because Google Cloud is too expensive to maintain.

Suppose you have the same local environment set up as I do, meaning having installed Ingress, Skaffold, and Docker globally. You can quickly run all services at once just by running this command.

skaffold dev

Week 4 - Branch name (feature/week-4)

This week I focus on improving the Micronaut services and creating first classes and packages that are going to shape the services. Also, I will prepare the environment for implementing databases. It will result in the first CRUD operations.

Spent time: ~9 hours

Challenges

• Creating relations between classes like domain entities or models
• Implementing Put operation and DTO
• Creating appropriate folder structure

Progress

I have created a folder structure for each service. We have packages such as Controller, DTO, Entity, Models, Repository, and Service. A controller is will have files providing endpoints and filter requests with middleware (I will implement it later). DTOs (Data Transfer Object) will be objects that define how the data will be sent over the network. Entities are going to map the tables that we will have in our database. In models is going to be stored data classes that shape the objects (It might be renamed to Types). In repositories, we will access data in the repository. Services will handle our business logic.

What did I learn?

Very slowly I am picking up this framework and implementing Kotlin language for using backend API service. I have learned the following:

• How to return data after a GET request.

@Controller("/api/registration")
class RegistrationController {

	var userData: User = User("1", "John", "Doe", "john@doe.com", "password", "123456789", "123456", "Player")
	val playerData = Player("1", "", "male", userData)
	val regitrationList = listOf<Registration>(Registration("1", "Tuesday", "Registered", "Competitive", playerData))

	@Get
	fun findAll(): List<Registration> {
		return regitrationList
	}

	@Get("/{id}")
	fun findById(@PathVariable id: String): Registration? {
		return regitrationList.find { it.id == id }
	}
}

• How to create data classes that will shape objects we will use such as User, Player, Team, etc.

data class User(
	val id: String,
	val firstName: String,
	...
	val userRole: String
)

data class Registration(
	var id: String,
	val matchDay: String,
	...
	val Player: Player,
)

Potential Improvements

• I have implemented temporary static data that response from the controller. This should be improved by implementing database.
• In controllers of both services are missing PUT and DELETE operations. It took me a little bit more reading of JDBC and Micronaut documentation. The answer is here. I will fix it the following week.
• Folder structure might change, meaning change name into Pascal case.
• Refactor code. Remove and re-consider duplicity of the model classes.

Week 5 - Branch name (feature/week-5)

This week I focused on changing the schema model based on the feedback, created two CRUD operations I left from the previous week, wired PostgreSQL with the Registration service and wrote the first migrations. All the changes were in Registration Service for this week.

Spent time: ~6 hours

Challenges

• Fix the DTO class for Update operation by Micronaut documentation and add Delete operation
• Connect JDBC and create a repository for for the Registration table
• Create RegistrationService and connect the CRUD operations from the controller to change records in the database.

Progress

The first thing I implemented was feedback from the last week. I have changed the ERD Model and changed the images. Then reduced the models, so I left only the model User and removed the models Coordinator and Player. I had reasons why I separated them. However, for this little project, I decided to simplify it.

Then I learned how to apply the DTO projection class by annotating with @Introspected. This helped me to finally solve my Update and add Create CRUD operation.

Once I solved controller operations I started working on RegisterService. The service will handle business logic and communicate with the repository that represents the database. The controller will only provide endpoints and validate requests.

RegistrationRepository.kt

@JdbcRepository
interface RegistrationRepository  : PageableRepository<Registration, UUID>

PageableRepository extends the basic CrudRepository that provides us simple operations with the database data such as findAll,findById, deleteAll etc.

This interface might be extended in the future. For example, if we want to delete all registration of particular user.

Also, we want we can override the vanilla operations CrudRepository provides by simply typing override before writing the function. registration service.kt

@Singleton
class RegistrationService(private val registrationRepository: RegistrationRepository) {

	...

	fun create(@Body body: CreateRegistrationDto): Registration {
		var newRegistration = Registration(
			UUID.randomUUID(), body.matchDay, body
				.status, body.proficiencyId, body.playerId
		)
		return registrationRepository.save(newRegistration)
	}

	fun delete(id: UUID): Boolean {
		if (registrationRepository.findById(id).isEmpty) return false
		registrationRepository.deleteById(id)
		return true
	}
}

We have to inject the repository to access the database in the service. Then we can call them and do the operations as I mentioned above.

What did I learn?

This week I learned a bit more about Micronaut. Its documentation is pretty robust, and after two weeks, I finally feel comfortable and orientated there. Even though the framework is pretty similar to what I know from NestJS, I still manage to learn new things. I didn't learn here much about Kotlin itself. I learned more about the Kotlin syntax I learned in AoC. However, now I know a lot about how to make a backend in a different language.

Week 6 - Branch name (feature/week-6)

This week I focused on team service that uses MongoDB, unlike registration service. Also, I cleaned the code and created a custom error handler.

Spent time: ~7 hours

Challenges

• Clean the code where needed
• Integrate MongoDB
• Create a custom error handler

Progress

The first thing I did was clean the service from unused models and restructured the DTOs for creating and updating data.

Then I integrated MongoDB. There I got stuck for an hour. The documentation says that we should incorporate MongoDB in application.yml like so:

mongodb: uri: mongodb://username:password@localhost:27017/databaseName

However, when I ran the app, it returned an unauthorized error. After a while of googling, I never saw that someone would try to connect directly to a database in MongoDB. I fixed it by accessing the database without a direct reference to a database and specified it in the repository instead.

The next issue I tackled was my database schema. I hadn't known that MongoDB integrates its own Id object, which can't be UUID. If I wanted to have a UUID property, I had to set it up separately, like so:

@MappedEntity
data class TeamEntity(
	@field:Id
	@field:GeneratedValue
	var id: String? = null,
	var teamId: String, // UUID property
	val name: String,
	val userId: String,
	val imageId: String,
	val matchDay: String,
	val season: String,
	val year: String,
	val playerLimit: String
)

Similarly to the previous week, I implemented a controller and a service. While developing the service, I noticed one thing. If we want to update or delete a record, we want to find the record first, if it exists.

Like so:

TeamService.kt

fun update(id: String, body: TeamDto): TeamEntity {
		val foundTeam = teamRepository.findById(id)

		if (foundTeam.isEmpty)
			throw ClassNotFoundException("Team with id $id was not found")

		...

}

If the client asks to edit a non-existing record, it responds with an error. The error message that Micronaut provides returns a Java object with status 500. We don't want to produce an HTTP status code error 500, which indicates that our server failed. We want to return a status error 400 that says Bad request.

Postman response:

Server log:

For this case, Micronaut has a guide on creating our custom error handler.

The error exception for not found a record in the Team collection in MongoDB might look like this:

class TeamNotFound : RuntimeException()

@Produces
@Singleton
@Requires(classes = [RuntimeException::class])
class TeamNotFoundHandler(private val errorResponseProcessor: ErrorResponseProcessor<Any>) :
	ExceptionHandler<TeamNotFound, HttpResponse<*>> {

	override fun handle(request: HttpRequest<*>, exception: TeamNotFound): HttpResponse<*> {
		return errorResponseProcessor.processResponse(
			ErrorContext.builder(request)
				.cause(exception)
				.errorMessage("Team not found")
				.build(), HttpResponse.badRequest<Any>()) //
	}
}

We create a new class and extend it with RuntimeException class. We override the handle function that indicates what to throw once the handler is called. ErrorContext builder helps to build it with easier.

val foundTeam = teamRepository.findById(id)

if (teamRepository.findById(id).isEmpty)
	throw TeamNotFound()

Once we use it, we no longer get the 500 server error when the client sends an invalid request.

However, it would be inefficient to create an exception for every type of error. So I made a global exception that can be used for any niche. It has two parameters, message, and httpResponse. Usage might look like this:

TeamService.kt

fun delete(id: String): Optional<TeamEntity> {
	if (!ObjectId.isValid(id))
		throw GlobalException("Invalid Object Id", HttpResponse.badRequest())

	...

}

What did I learn?

This week I learned how to integrate MongoDB and what obstacles I might have encountered. Also, I learned that I could not use UUID for MongoDB. The most significant improvement was implementing error handlers. This was the critical thing I learned this week. It makes my responses with the client side neater. Lastly, I briefly developed User service in TypeScript.

Week 7 - Branch name (feature/week-7)

This week I focused on implementing NATS, cleaning up the services and integrating databases within the cluster. In the end, it project should be executable by simply typing scaffold dev and the whole project should run concurrently.

Spent time: ~16 hours

Challenges

• Learn how to implement NATS (no prior experience)
• Write deployment YAML for inner cluster databases and NATS server
• Clean up code (using enum classes and extending the number of error handlers)

Faced issues

I have to start with an issue I faced when I tried to implement NATS streaming server into my project. I followed this article from 2020. However, the solution I wanted to use is going to be deprecated in June 2023. Even though it's not June 2023 yet, Micronaut does not support NATS streaming server anymore and only supports JetStream which is going to replace. It took me a significant portion of time to find it out.

Progress

Despite, I haven't developed the NATS solution I can demonstrate how the technology works at least. I exposed my testing environment that tests communication through NATS. There are two main functions that every service needs to have Publisher and Listener. From the names, you can guess which one does what. One service publishes data to the broker and the second listens and handles the data. The communication works on the subscription principle. Similar to Observable in JavaScript.

If you want to test the behavior locally, please, text me. The process isn't the simplest one.

However, let's jump back to Micronaut. After a little failure that was caused by my unpreparedness, I jumped on improving my Kubernetes cluster. I implemented database pods that will connect to our Micronaut services, so we can run our cluster at once.

Then I cleaned up the naming convention for entities. That means, every entity holds its properties as a model class, so it doesn't need to be named as TeamEntity.

Then I refactored the code by adding enum classes and more error handlers for more use cases.

What did I learn?

I learned a lot about NATS. It took me over 6 hours to go through many videos on YouTube and articles on Medium. The most important this I found was how to handle asynchronous communication between services through NATS. There must be implemented version system so that the events won't skip each other. Then I learned a bit more about Micronaut and how to implement NATS. Even though I have implemented, yet, now I know what to focus on. Implementing error handlers, enums, and cleaning code I don't consider as learning new stuff but I think it was a good practice.