Recommend some worthy of learning open source projects and frameworks

Today, I received a question from a friend of the ball, let recommend some open source projects worth looking at, I think netty these are too complicated not too good to get started.

It's true that Spring and Netty, which we commonly use on a daily basis, do seem to be a bit of a head-scratcher due to the fact that they've been in development for many years.

I'm going to recommend some projects that I've seen and thought were good (almost all of them I've worked on), from easy to hard, which will also include Java and Go projects, mainstream middleware and cloud-native projects.

Java Project

xxl-job

Difficulty: 🌟🌟🌟🌟🌟🌟🌟🌟
Recommendation index：🌟🌟🌟🌟

xxl-job is a very classic scheduling framework, currently on GitHub also has 27k star attention, because the function is not complex so recently there is not much update.

Everyone uses this type of scheduling framework on a daily basis, so the difficulty of understanding it is very low, plus his implementation is relatively simple, for example:

• Using MySQL's Locks to Solve Distributed Locking Problems in a Simple and Brute-Force Way
• The use of thread pooling: Because each task needs to be scheduled in such a way that they don't affect each other as much as possible, there's heavy use of thread pooling in there, as well as some best practices on how to get the results of asynchronous tasks.
• RPC calls: there's a built-inRPC framework, also written by the author, which is not complicated to realize the principle, it is recommended to look at the source code, you can better understand our work in the rpc framework.

cim

Difficulty: 🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟
Recommendation index：🌟🌟🌟🌟
Here's a little bit of private stuff sandwiched between one of my own open sourceDistributed Instant Messaging System, actually the last version of the code was pretty badly written now that I look at it, but it's good to see that the recent release of v2.0.0 has improved the quality of the code quite a bit.

It has the basic functions of IM instant messaging, and at the same time it can be realized based on it:

• instant messaging (IM)
• push message
• IOT Messaging Platform

With cim you can learn to be in a distributed system:

• How metadata is stored and synchronized.
• How RPC calls are implemented.
• How the long link system is implemented.
• How to do integration testing for complex distributed systems, etc.

For a detailed description of the program you can check the home page of thereadmeIf you find something that needs to be optimized (there are actually quite a few todo's that you haven't done yet), you are welcome to submit a PR.

PowerJob

Difficulty: 🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟
Recommendation index: 🌟🌟🌟🌟
PowerJob Is also a scheduling framework, only he has the advantage of latecomers, combined with the advantages of other scheduling systems on the market at the same time also added some new features, the following is the official comparison of his features:

The community is also much more active compared to xxl-job, and has just released the5.1.0 version, while the community organizes many learning articles andparticulars：

It uses Akka to implement remote communication, and those interested in this section should not miss it to see some best practices.
The code in it is also written in a very standardized way, and some of the classes are well designed and extensible, such as the common actuators that are all accessible via a
MapProcessor Expanded.

We recommend that you start with the task scheduling piece:#onReceiveServerScheduleJobReq

Pulsar

Difficulty: 🌟🌟🌟🌟
Recommendation index: 🌟🌟🌟🌟

Pulsar is now the dominant cloud-native message queuing middleware and is now used by a large number of companies, through whom you can learn:

• API design: Pulsar's client is directly oriented to developers, and each iteration of upgrades has to take compatibility into account while maintaining ease of use.
• Asynchronous calls: Almost all requests in Pulsar are asynchronous, so there is heavy use of asynchronous ➕ callbacks (although there are some pitfalls), and you can learn a few things about how to write high-performance code.
• Best use of Netty: The underlying network framework for sending and receiving messages is also underpinned by Netty and encapsulated by Pulsar.
• Protocol-based multilingual client.
  • Because Pulsar's communication codec uses protocol, it is possible to generate APIs for various languages based on it, so it is very convenient to write clients for other languages based on it.

However, due to the complexity of Pulsar itself, the threshold for getting started is still not low, and it is recommended to start with client-side code (both Java and Go).

StarRocks

Difficulty: 🌟🌟🌟🌟🌟
Recommendation index: 🌟🌟🌟🌟

StarRocks is also an OLAP database project that I've only recently been introduced to, and I've learned from scratch, having accumulated almost zero knowledge of the field before.

It's a good thing that I've submitted a few PRs to it during this period of time because I've had the need to do so, and I've gradually become familiar with it.

I contact down these open source projects, found that StarRocks such database project is the most promising (money) prospects, after all, and data to deal with the product company's willingness to pay will be a little higher.

However, the project is really not very friendly to newcomers, it is best to have been exposed to the field of big data and then learn will be more appropriate, but do not be afraid, I am a pure white, no foundation to follow the code debug, anyway, are written in Java can always understand.

Here's a recommended first look at what I wrote earlierSetting up a local development environmentThis way you can debug it in idea.

OpenTelemetry

Difficulty: 🌟🌟🌟🌟
Recommendation index: 🌟🌟🌟🌟🌟

OpenTelemetry now serves as the de facto standard for cloud-native observability, and is now progressively becoming a must-have technology stack for major companies.

By means of ajavaagent You can automatically collect trace, metrics, logs, and other data from your application.opentelemetry-java-instrumentationBecause it's based on this program that we use most of our day-to-day packagedjavaagent, through which one can learn:

• How to write interceptors for arbitrary functions
• How trace information is passed between threads and processes
• How some common frameworks work
  • For example, you need to understandgRPC You can check how OpenTelemetry buried points against him to know the core principles of his.
• Elegant API Design

Meanwhile OpenTelemetry is kind of one of the most elegant codes I've ever seen, and I highly recommend that everyone take a look at it.

If you're not familiar with OpenTelemetry, you can start by checking out myPreviously written。

Go (cloud-native project)

cprobe

Difficulty: 🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟
Recommendation index：🌟🌟🌟🌟

cprobe is an observability item, and his purpose is to be able to integrate all kinds of exporters, such as thekafka_exporter, nginx_exporter, mysql_exporter etc.

Upper abstraction is also done to unify the management of the configuration of various monitoring objects so that one process can be deployed to monitor all applications.

There are lessons to be learned through this program:

• Fundamentals of monitoring systems, such as Prometheus and metrics
• Basic Usage of the Go Language

I've written about this before.Hands-on with contributing code to open source projectsIt is cprobe that is used as an example to introduce.

VictoriaLogs

Difficulty: 🌟🌟🌟🌟
Recommendation index: 🌟🌟🌟🌟

It's a part ofVictoriaMetrics A sub-project, through the name should know that he is mainly used for processing logs, you can understand him as a simple version of ElasticSearch, although the function is simple, but the resource consumption will be much lower than the ES, you can see the following pressure test chart:

There are lessons to be learned through this program:

• How data is stored and queried on disk
• The Go language has a lot to say about thegoroutine cap (a poem)channel Some best practices
  The current version is still relatively early, so none of the code is too complex, and it is recommended that you can start with the entrance to the querylook up。

summarize

All of the above are programs I've had serious exposure to, so if you're looking to plow through a long period of time while screwing around with money, I'd recommendStarRocks, which is also hot at the moment.

If you just want to level up in Java, then Pulsar and OpenTelemetry are recommended, both have a lot of code best practices.

If you want to get into cloud-native and Go projects, cprobe is the way to go.

Of course, no matter which project is the most important thing is to adhere to, a lot of projects if you just occasionally look at it is easy to forget, at least to do really run up and debug the code.

Reference Links:

• /powerjob/guidence/wu2e93
• /VictoriaMetrics/VictoriaMetrics/blob/master/lib/logstorage/storage_search.go
• /tags/OpenTelemetry/
• /2024/10/09/ob/StarRocks-dev-env-build/