Preface
This is the second article in my "Looking Back on My Development Experience" series, focusing on my story with code generators. From the initial simple tool to the final evolution into a powerful ORM framework, this experience not only greatly improved my technical skills, but also gave me a deep understanding of the importance of refactoring, design patterns, and system architecture. I hope that this article can provide you with some inspiration and thoughts.
First introduction to code generators
After working for a few years, I came across code generators for the first time. Although the initial version was very simple, the idea really stood out to me. In the next dozen years, I continued to expand and improve this code generator, and finally realized the automatic generation of a website with complete addition, deletion, modification and query functions through the database. I learned a lot from this experience and I summarize some of the key points in this blog post:The evolution of code generators。
Code Generator and Reading
When I first started writing code generators, my understanding of object-oriented development was still vague. At that time, I was reading "C# Primer Plus" and applying the knowledge in the book to the code generator while studying. Sometimes I encounter a problem and don't know how to solve it. Sometimes I learn new knowledge points and can practice them in the code generator immediately. This experience was so amazing that I often thought about the content in the book while walking, so much so that I missed the intersection many times. I am here"my reading career” reviewed this experience.
In this process of learning and practicing, the code generator gradually changed: its functions continued to expand, the amount of code decreased, but the inheritance relationship became complicated. I even designed a base class for business objects, which I later discovered was a misunderstanding. Nonetheless, the code generator evolved into a framework—a framework of my own. The size of the code generator itself and the generated code are significantly reduced, but it is more powerful and maintainable.
The birth of the ORM framework
Sometimes, change comes from a casual word. While discussing with colleagues, someone mentioned that "SQL statements should not be placed in logic code". This sentence touched me, and I started to remove SQL statements from the code in the code generator, which laid the foundation for the later formation of the ORM framework.
Another technical source of ORM framework is the C# open source project DNN (Dot Net Nuke). i learned from itCBO(Object creation and assignment based on metadata), this has become one of the core technologies of the ORM framework. For example, convert a Row in a DataTable to an object, or convert an object to a DataRow;There is also the Null class. also,Strategy pattern in DNNIt also provided me with inspiration to enable the ORM framework to support multiple databases.
Between 2006 and 2007, I implemented an ORM framework similar to MyBatis, and I even felt that it was more useful than MyBatis (I was ashamed).
XML reading and database access
In the ORM framework, I put SQL statements and parameters into XML, and designed an access framework to implement caching and update functions. in order to supportVarious databases, I applied design patterns even though my understanding of design patterns was not deep enough at the time. Later I realized that design patterns are not learned deliberately, but naturally "grow" in practice. I refactored the database access program many times and ended up with a stable and efficient library.
Detours in technological pursuit: Attributes
During this experience, I also took some detours. For example, I overusedAttributes in C#, or even put it into a business object class. I later realized this was wrong and removed it. This experience made me understand that the use of technical tools needs to be moderate, and the maintainability of the code cannot be sacrificed for the sake of showing off skills.
Development of front-end controls
In the early days of code generators, I mainly did web development and front-end use, and later moved to Castle Monorail. In order to reduce the amount of code generated on the web page and backend, I continue to expand the web control, put as many front-end processing functions as possible on the front end, and abstract common functions. The abstraction of front-end controls has gone through two stages:
-
first stage: Abstract web controls containing background code, such asExtend DataGrid andTextBox control。
-
second stage: Use pure front-end controls to further improve flexibility and maintainability.
Front-end automatic input legality verification framework
In addition to extending the control, I also designed a front-end input legality verificationframe. Traverse all DOM controls through JavaScript, conduct legality verification based on extended attributes (such as whether it is empty, maximum and minimum length, integer, etc.), and pop up a prompt message below the control. This design greatly reduces the burden of back-end verification.
Use template technology
Initially, all code for the code generator was generated via string concatenation. With the upgrade of the framework, the amount of code generated is reduced, and the workload of string concatenation is also significantly reduced. Later, I realized that I could use templates to generate code, so I designed a simple template engine that replaced variables with the "#***#" format. Later, I introduced template technologies such as VTL to further reduce the amount of code in the code generator.
Excel operations
I added Excel import and export functions to the framework, generate Excel files based on the database structure, and support data legality verification and formatting. Users can edit data in Excel and then import it into the system to add data in batches while strictly following the primary and foreign key relationships to avoid dirty data. This feature played a huge role in the subsequent video surveillance configuration, shortening the configuration work that originally took several days to more than ten minutes.
Frameworks and Libraries
After several years of development, I discovered that a framework does not necessarily exist in a single form, but can be split into multiple independent libraries. Each library can be used individually or combined to form a complete framework. Based on this idea, I split the framework into data access library, metadata mapping library (CBO), SQL cache library, front-end legality check library, front-end control library, Excel import and export library, and general library.
Code generator extension
The code generator was initially a simple Form page, and later gradually expanded to an interface containing multiple Panels, supporting database table trees, code generation areas, database table structure selection, and code type options. The generated code not only supports Java, C++, Castle Monorail page code and multi-language configuration.
Code generators and refactoring
The habit of refactoring is gradually developed in the process of writing code generators. From the initial mess of code to a complete framework and then split into multiple libraries, refactoring played a key role. This experience made me form several important insights:
-
When refactoring, you need to take small steps and run quickly. If you take too big a step, you will lose your way.
-
The more you refactor the code, the more you dare to touch it; otherwise, the code will gradually become rigid.
-
After several small refactorings, a relatively complete verification is required to avoid wasting time.
-
The goal of refactoring can be larger, but it should start with small steps and gradually achieve it.
-
The ultimate goal of refactoring is to make the code clearer and more flexible.
Follow-up
I have been writing this code generator for more than ten years, enjoying it and learning a lot. I have also implemented similar tools in Python and Java, such as the Python version of the code generator (Mysql_CodeGen) and the Java version of the framework (HiJDB). In addition, I also developed a C# version of the DataGridView auxiliary class (DataGridView helper class）。
Summarize
By writing code generators, my technical skills have been greatly improved, and I have a deep understanding of refactoring, object-oriented, design patterns, front-end technology, and back-end technology. The only regret is that this experience is mainly done alone. If you can cooperate with more people, you may grow faster and gain more. I hope to find similar opportunities again in my future work. Just like my code generator, I started from scratch, from framework to library, constantly refactored and made breakthroughs, and gradually improved myself.
In fact, in terms of development, I did encounter similar work again, such as ACE, such as the smart campus platform that I later developed.
Just like my code generator, from no framework to having a framework to breaking the framework to form multiple libraries, I hope that in any good or bad situation, I will have the courage to reconstruct myself, in a positive and negative spiral, and gradually Complete.