Objectives of the chapter
- Completion of the basic design of the data access layer
- Implementing a RESTful API for Sticker Microservices
Introduction: should you use an ORM framework?
It goes without saying that Sticker microservices need access to a database to manage the "stickers" (aka "Stickers"), so there is no getting around the question of how to store the data. If you follow the idea of domain-driven design, then you can say that the data stored in the database is the "sticker".polymerizationAfter persisting to thestore in a warehouseafter an object state. So now the question is, do we need to follow the idea of domain-driven design?
In the current design and implementation of the Sticker microservice, I think for the time being it should not be needed, the main reason is that the business here is not complex, at least in the Bounded Context of the Sticker microservice, which focuses on the Sticker object, and the behavior of this object is very simple, and it can even be seen as a simpleData Transfer Objects(DTO), its role is just to save the "sticker" data in a structured way. You may wonder, if the business expands in the future, will you still consider introducing some domain-driven design implementation ideas or even related design patterns? I think the answer is: it is possible, but as far as Sticker microservices are concerned, unless there are more complex business functions that need to be implemented, it may be a better choice to keep the simplicity and lightness of Sticker microservices. Almost 3 years ago, I summarized an article, theWhen to use domain-driven designThe "domain-driven design", for the domain-driven design related content to do a summary of the summary, interested readers are welcome to move to read.
So, for now, we will look at the Sticker object as a "data transfer object" and not as an aggregate. Since we will choose PostgreSQL as the database for our backend, which is a relational database, we return to the question in the title: should we use an ORM framework? I don't think it's necessary, because we don't intend to deal with "Stickers" from the perspective of a business object, and the object structure of "Stickers" itself is very simple, probably only a few attribute fields, so maybe it would be more lightweight to use it directly, even if Even if the "sticker" object contains a simple hierarchical structure, the use of the implementation will not be particularly troublesome. On the other hand, the use of ORM has a certain cost, not only in the efficiency of code execution, in the ORM configuration, code programming model, model mapping, database initialization, and model version migration, etc., there will be some additional overhead, for Sticker microservices, there is not much need.
To summarize, at present we will not introduce too much domain-driven design thinking, nor will we use a certain ORM framework to do data persistence, but will design a relatively simple data access layer and combine it to achieve the data access of Sticker microservices. This level of interface is well defined, and in the future, if the business logic is extended and the model objects are complex, it is not impossible to hopefully reintroduce ORM.
Basic design of the data access layer
In Sticker microservices, I introduced something called a "Simplified Data ACcessor", which provides the caller with the ability to add, delete, modify, and retrieve business entity objects. Specifically, it will contain at least the following methods:
- Save the given entity object to the database (incremental)
- Remove a given entity object from the database (delete)
- Updating entities in the database (change)
- Get the entity object based on the ID of the entity (check)
- Based on the given paging method and filtering conditions, return the entities of a page that satisfy the filtering conditions (check)
- Based on the given filter condition, return whether the entity that satisfies this filter condition exists (check)
In the later implementation of the Sticker microservice API, we will use this SDAC to access the back-end database to realize the management of "stickers". According to the above analysis, it is not difficult to dig a technical need, that is, in the future there may be a need to introduce ORM to achieve data access, although we will not do so in the short term, but in the beginning, set the general direction of the design, is always a better practice. Thus, it also leads to a basic idea of SDAC design: define the interface, and then implement SDAC based on PostgreSQL, and then in the Core Web API, use dependency injection to inject the PostgreSQL implementation into the framework, so the API controller only needs to rely on the interface of the SDAC can be, and in the future, it will be more convenient to replace different implementations. in the future, it will also be more convenient to replace different implementations.
In this chapter we don't do the PostgreSQL implementation, which is left to be introduced in the next lecture, and in this chapter we implement a simple SDAC based only on an in-memory list data structure, since the focus of the discussion in this chapter is actually the API implementation in the Sticker microservice. Obviously, this also benefits from interface-oriented abstraction design ideas. To summarize, the SDAC-related objects and the relationships between them will look roughly like the following:
First, define aISimplifiedDataAccessor
interface, which is placed in a separate package (.NET in Assembly)This interface defines a set of basic CRUD methods in a separate package under
in which there is a class that implements the interface:
InMemoryDataAccessor
, which contains aIEntity
entity's list data structure, and then based on that list, implement theISimplifiedDataAccessor
under all the methods. And theThe API controller in the
StickersController
dependencyISimplifiedDataAccessor
interface, and the Core's dependency injection framework puts theInMemoryDataAccessor
instance is injected into the controller.
The parameters and return types of all methods in the class diagram have been simplified for the sake of aesthetics of composition, and in the code of the case, the parameters and return types of the individual methods are slightly more complex than shown in the diagram.
Here we introduce theIEntity
interface, which needs to be implemented by all data objects that can access data through SDAC. An important purpose for introducing this interface is to implement generic constraints so that theISimplifiedDataAccessor
The interface explicitly specifies what kind of objects can be used for data access. Also, a generic type is introduced here:Paginated<TEntity>
type, which can contain paging information and where theItems
attribute holds the data for a particular page (the page number is defined by thePageIndex
attribute is specified), as in theStickersController
In the controller, we will probably need to implement a "sticker" query that can support paging.
Space constraints preventedInMemoryDataAccessor
In the specific implementation of each method is introduced, if interested, you can open the source code link posted at the end of this article, directly open the code to read. Here we focus on the interpretation of theGetPaginatedEntitiesAsync
method's code:
public Task<Paginated<TEntity>> GetPaginatedEntitiesAsync<TEntity, TField>(
Expression<Func<TEntity, TField>> orderByExpression, bool sortAscending = true, int pageSize = 25,
int pageNumber = 0, Expression<Func<TEntity, bool>>? filterExpression = null,
CancellationToken cancellationToken = default) where TEntity : class, IEntity
{
var resultSet = filterExpression is not null
? _entities.Cast<TEntity>().Where(())
: _entities.Cast<TEntity>();
var enumerableResultSet = ();
var totalCount = ;
var orderedResultSet = sortAscending
? (())
: (());
return (new Paginated<TEntity>
{
Items = (pageNumber * pageSize).Take(pageSize).ToList(),
PageIndex = pageNumber,
PageSize = pageSize,
TotalCount = totalCount,
TotalPages = (totalCount + pageSize - 1) / pageSize
});
}
The purpose of this method is to return the entity data for a particular page, first of all paging is needed based on sorting, so theorderByExpression
The parameter specifies the sorted fields via a Lambda expression;sortAscending
It is well understood that it specifies whether to sort in ascending order or not;pageSize
cap (a poem)pageNumber
Specify the number of data records per page and the page number of the data to be returned when paging; pass thefilterExpression
Lambda expression parameter, you can also specify the query filter, for example, only return the "Creation Date" is greater than a certain day data. In theInMemoryDataAccessor
are operating directly on the list data structure, so the implementation of this function is still relatively simple and easy to understand: if thefilterExpression
is defined, then the filtering operation is performed first, followed by sorting and constructing the Paginated<TEntity> object as the return value of the function. We'll see another different implementation of this function in the next article on PostgreSQL data access implementations.
GetPaginatedEntitiesAsync is an asynchronous method in terms of the interface definition, so we should pass in the CancellationToken object if possible to be able to support the cancel operation in that method.
Now that we have the data access layer in place, we can start implementing the RESTful API for Sticker microservices.
StickersController
We are using Core Web API to create the StickersController controller, so we will also default to using RESTful to implement the microservices API, RESTful API based on the HTTP protocol, is currently one of the most widely used protocols for communication between microservices, because it is mainly based on the JSON data format, so it is particularly friendly to the front-end development and implementation. RESTful for the data being accessed under the unified view as a resource, is a resource on the address, supported access methods and other attributes, but here we do not discuss these contents in depth, focusing on a few key points of the implementation of StickersController.
Injection of ISimplifiedDataAccessor
Readers familiar with Core Web API development should be very familiar with how to inject a Service, here is a brief introduction. In theproject-based
file, add the following line of code directly, note that before adding, first add to the project on the
Project Citation:
<ISimplifiedDataAccessor, InMemoryDataAccessor>();
Here, I willInMemoryDataAccessor
It is registered as a singleton instance, and although it is a stateful object, the purpose of using it is just to get the whole application up and running, and it is going to be replaced with PostgreSQL later on (PostgreSQL's data access layer is stateless, so it makes sense to use a singleton here), so there is no need to get hung up on whether it makes sense to implement it on its own, and whether or not it is thread-safe in a singleton instance. So there's no need to worry about whether it's a reasonable implementation, or whether it's thread-safe under a single instance. The design principle of high cohesion and low coupling makes the problem much simpler.
will nowunder the item
WeatherForecastController
Delete it and add a new Controller namedStickersController
The basic code structure is as follows:
namespace ;
[ApiController]
[Route("[controller]")]
public class StickersController(ISimplifiedDataAccessor dac) : ControllerBase
{
// Other codes are omitted for the time being
}
So it's possible to make a difference in theStickersController
In the controller, thedac
instance to access the data store now.
Controller code testability: since StickersController only relies on the ISimplifiedDataAccessor interface, it is entirely possible to generate a Mock object of ISimplifiedDataAccessor through Mock technology and then inject it into the StickersController to complete the unit tests.
Return reasonable HTTP status codes in controller methods
It is a best practice in RESTful API development that for different RESTful APIs, reasonable HTTP status codes should be returned in different situations. Especially in microservice architecture, reasonable definition of API return code is good for multi-service integration. I think the following principles can be followed:
- Try to avoid directly returning 500 Internal Server Error
- If the API cannot be executed successfully due to non-compliance of the incoming data from the client, a status code (4XX) starting with "4" should be returned, for example:
- If the client sends a resource query request, but the resource does not actually exist, 404 Not Found is returned.
- If the resource you wish to create already exists, you can return 409 Conflict
- If there is a problem with some of the data in the resource passed in by the client, a 400 Bad Request can be returned.
- The POST method is usually used to create a new resource, so it usually returns 201 Created, and in the response body, the address of the newly created resource is specified. Of course, there are cases where POST is not used to create a new resource, but to perform a task, which can also be returned as 200 OK or 204 No Content.
- The PUT, PATCH, and DELETE methods determine whether they should return 200 OK or 204 No Content, depending on whether they need to return resource data.
Take the following three RESTful API methods as an example:
[HttpGet("{id}")]
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(Sticker))]
[ProducesResponseType(StatusCodes.Status404NotFound)]
public async Task<IActionResult> GetByIdAsync(int id)
{
var sticker = await <Sticker>(id);
if (sticker is null) return NotFound($"Sticker with id {id} was not found.");
return Ok(sticker);
}
[HttpPost]
[ProducesResponseType(StatusCodes.Status201Created)]
[ProducesResponseType(StatusCodes.Status409Conflict)]
[ProducesResponseType(StatusCodes.Status400BadRequest)]
public async Task<IActionResult> CreateAsync(Sticker sticker)
{
var exists = await <Sticker>(s => == );
if (exists) return Conflict($"Sticker {} already exists.");
var id = await (sticker);
return CreatedAtAction(nameof(GetByIdAsync), new { id }, sticker);
}
[HttpDelete("{id}")]
[ProducesResponseType(StatusCodes.Status204NoContent)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
public async Task<IActionResult> DeleteByIdAsync(int id)
{
var result = await <Sticker>(id);
if (!result) return NotFound($"Sticker with id {id} was not found.");
return NoContent();
}
These methods all use theSticker
This class represents the "sticker" object, which is actually a domain object, but as mentioned above, we will only use it as a data transfer object for now, and it is defined as follows:
public class Sticker(string title, string content) : IEntity
{
public int Id { get; set; }
[Required]
[StringLength(50)]
public string Title { get; set; } = title;
public string Content { get; set; } = content;
public DateTime CreatedOn { get; set; } = ;
public DateTime? ModifiedOn { get; set; }
}
Sticker
class implements theIEntity
interface, which isA class in the project that is defined in the
project, rather than being defined in the
project because from the perspective of Bounded Context's division it is
An internal business object of the project that is not used by other microservices.
existCreateAsync
method, it will first determine whether the "sticker" with the same title exists, if it does, then return 409; otherwise, it will directly create the sticker and return 201, along with the address of the "sticker" resource after the successful creation (CreatedAtAction
method indicates that the resource was created successfully and can be accessed via theGetByIdAsync
method, with the Id of the new "Stickers" resource). TheDeleteByIdAsync
method, the API will directly try to delete the "sticker" of the specified Id, if the sticker does not exist, it will return 404, otherwise it will be successfully deleted and return 204.
Incidentally, the methods used on the variousProducesResponseType
Attribute, in general, we can be able to return the HTTP status code of the current API method are marked with this attribute (Attribute), so that Swagger can generate more detailed documentation:
Model Validation in Core Web API
Core Web API is able to automate model validation before a controller method is called. For example, in the CreateAsync method above, why didn't I return null for the Title field of the Sticker? Why did I not return null for the Title field in the Sticker method above, when the return status definition of the API clearly states that it is possible to return 400?Required
cap (a poem)StringLength
These two characteristics:
[Required]
[StringLength(50)]
public string Title { get; set; } = title;
So, when the Sticker class is used in the POST request body of the RESTful API, the Core Web API framework automatically completes the validation of the data model based on these characteristics, for example, by executing the following command after starting the program:
$ curl -X POST http://localhost:5141/stickers \
-d '{"content": "hell world!"}' \
-H 'Content-Type: application/json' \
-v && echo
will get the following return result:
Not only that, but developers can extendto implement custom validation logic.
PUT or PATCH?
In the development of RESTful API, there is a more entangled problem is, in the modification of resources, is it should be PUT or PATCH?In fact, it is very simple, the definition of PUT is: use the same data of another resource to replace the existing resources, while the PATCH is to modify the existing resources for a particular . Therefore, from the point of view of modifying the object alone, PATCH is more efficient than PUT, it does not require the client to modify the object needs to be downloaded in its entirety, after modifying the entire nature of the back-end sent to save. Thus, another problem arises: how does the server know which attribute field of the resource should be modified and what is the way of modification? A more direct approach is that the server still receives from the client by the PATCH method sent over the Sticker object, and then determine whether the value of each field in this object has a value, if so, it means that the client wants to modify the field, otherwise it will skip the modification of this field. If the object structure is relatively simple, this approach may also be okay, but if the object contains a large number of attribute fields, or it has a certain hierarchical structure, then this approach will be clumsy, not only time-consuming and laborious, but also error-prone.
A better practice in RESTful API implementations is to use theJSON PatchIt is a set of international standards (RFC6902), which defines the basic format and specifications for modifying JSON documents, and Microsoft's Core Web API.Native support for JSON Patch. The following is the method of using JSON Patch in the StickersController controller:
[HttpPatch("{id}")]
[ProducesResponseType(StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
[ProducesResponseType(StatusCodes.Status400BadRequest)]
public async Task<IActionResult> UpdateStickerAsync(int id, [FromBody] JsonPatchDocument<Sticker>? patchDocument)
{
if (patchDocument is null) return BadRequest();
var sticker = await <Sticker>(id);
if (sticker is null) return NotFound();
= ;
(sticker, ModelState);
if (!) return BadRequest(ModelState);
await (id, sticker);
return Ok(sticker);
}
The logic of the code is very simple, first get the "sticker" object by Id, then use themethod, which applies the client's modification request to the sticker object, then calls SDAC to update the data in the back-end storage, and finally returns the modified sticker object. Let's test this by first creating a new sticker:
$ curl -X POST http://localhost:5141/stickers \
> -H 'Content-Type: application/json' \
> -d '{"title": "Hello", "content": "Hello daxnet"}' -v
Note: Unnecessary use of -X or --request, POST is already inferred.
* Host localhost:5141 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
* Trying [::1]:5141...
* Connected to localhost (::1) port 5141
> POST /stickers HTTP/1.1
> Host: localhost:5141
> User-Agent: curl/8.5.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 45
>
< HTTP/1.1 201 Created
< Content-Type: application/json; charset=utf-8
< Date: Sat, 12 Oct 2024 07:50:00 GMT
< Server: Kestrel
< Location: http://localhost:5141/stickers/1
< Transfer-Encoding: chunked
<
* Connection #0 to host localhost left intact
{"id":1,"title":"Hello","content":"Hello daxnet","createdOn":"2024-10-12T07:50:00.9075598Z","modifiedOn":null}
Then, check to see if the data for this sticker is correct:
$ curl http://localhost:5141/stickers/1 | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 110 0 110 0 0 9650 0 --:--:-- --:--:-- --:--:-- 10000
{
"id": 1,
"title": "Hello",
"content": "Hello daxnet",
"createdOn": "2024-10-12T07:50:00.9075598Z",
"modifiedOn": null
}
Now, use the PATCH method to change the content to "Hello World":
$ curl -X PATCH http://localhost:5141/stickers/1 \
> -H 'Content-Type: application/json-patch+json' \
> -d '[{"op": "replace", "path": "/content", "value": "Hello World"}]' -v && echo
* Host localhost:5141 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
* Trying [::1]:5141...
* Connected to localhost (::1) port 5141
> PATCH /stickers/1 HTTP/1.1
> Host: localhost:5141
> User-Agent: curl/8.5.0
> Accept: */*
> Content-Type: application/json-patch+json
> Content-Length: 63
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Date: Sat, 12 Oct 2024 07:56:04 GMT
< Server: Kestrel
< Transfer-Encoding: chunked
<
* Connection #0 to host localhost left intact
{"id":1,"title":"Hello","content":"Hello World","createdOn":"2024-10-12T07:50:00.9075598Z","modifiedOn":"2024-10-12T07:56:04.815507Z"}
Note that the above command requires theContent-Type
set toapplication/json-patch+json
, perform another GET request to verify it:
$ curl http://localhost:5141/stickers/1 | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 134 0 134 0 0 43819 0 --:--:-- --:--:-- --:--:-- 44666
{
"id": 1,
"title": "Hello",
"content": "Hello World",
"createdOn": "2024-10-12T07:50:00.9075598Z",
"modifiedOn": "2024-10-12T07:56:04.815507Z"
}
As you can see, the content has been changed to Hello World, while themodifiedOn
field is also updated to the UTC time at which the current resource was changed.
If you need to store time information on the server side, you should generally save it as UTC time, or local time + time zone information, so that you can also infer the UTC time, in short, on the server side, you should take the UTC time as a standard, so that the clients in different time zones can calculate and display the local time based on the UTC time returned by the server side, and there will be no confusion.
Using JSON Patch in Core also requires the introduction of the Newtonsoft JSON Input Formatter, as described in theSteps to Microsoft's Official DocumentationJust make the settings.
Support for Sorted Field Expressions on the Paged Query API
In the front-end application, usually can support user-defined data sorting, that is, the user can decide which field of the data is sorted in ascending or descending order, and then based on such sorting to complete the paging function. In fact, the basic principle of the realization I have already in theDynamically Building Lambda Expressions to Sort Data by Specified Fields on Core Web APIsThe idea is to build a Lambda expression based on the input field name, and then apply the Lambda expression to the OrderBy/OrderByDescending method of the object list, or to the database access component to realize the sorting function. Here is the relevant code in the StickersController controller:
[HttpGet]
[ProducesResponseType(StatusCodes.Status200OK)]
public async Task<IActionResult> GetStickersAsync(
[FromQuery(Name = "sort")] string? sortField = null,
[FromQuery(Name = "asc")] bool ascending = true,
[FromQuery(Name = "size")] int pageSize = 20,
[FromQuery(Name = "page")] int pageNumber = 0)
{
Expression<Func<Sticker, object>> sortExpression = s => ;
if (sortField is not null) sortExpression = ConvertToExpression<Sticker, object>(sortField);
return Ok(
await (sortExpression, ascending, pageSize, pageNumber)
);
}
private static Expression<Func<TEntity, TProperty>> ConvertToExpression<TEntity, TProperty>(string propertyName)
{
if ((propertyName))
throw new ArgumentNullException($"{nameof(propertyName)} cannot be null or empty.");
var propertyInfo = typeof(TEntity).GetProperty(propertyName);
if (propertyInfo is null) throw new ArgumentNullException($"Property {propertyName} is not defined.");
var parameterExpression = (typeof(TEntity), "p");
var memberExpression = (parameterExpression, propertyInfo);
if ()
return <Func<TEntity, TProperty>>(
(memberExpression, typeof(object)),
parameterExpression);
return <Func<TEntity, TProperty>>(memberExpression, parameterExpression);
}
The following shows the command line and API call output in descending order based on the Id field:
$ curl 'http://localhost:5141/stickers?sort=Id&asc=false&size=20&page=0' | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 453 0 453 0 0 205k 0 --:--:-- --:--:-- --:--:-- 221k
{
"items": [
{
"id": 4,
"title": "c",
"content": "5",
"createdOn": "2024-10-12T11:55:10.8708238Z",
"modifiedOn": null
},
{
"id": 3,
"title": "d",
"content": "1",
"createdOn": "2024-10-12T11:54:37.9055791Z",
"modifiedOn": null
},
{
"id": 2,
"title": "b",
"content": "7",
"createdOn": "2024-10-12T11:54:32.4162609Z",
"modifiedOn": null
},
{
"id": 1,
"title": "a",
"content": "3",
"createdOn": "2024-10-12T11:54:23.3103948Z",
"modifiedOn": null
}
],
"pageIndex": 0,
"pageSize": 20,
"totalCount": 4,
"totalPages": 1
}
Tip: Use lowercase naming conventions in URLs
Since C# programming specifies the use of the Pascal naming convention for identifiers, and the Core Web API generates URLs based on the names of Controllers and Actions, the Pascal naming convention is used by default in paths, which means that the first character is an uppercase letter. For example: http://localhost:5141/Stickers, where "SThe "S" in "tickers" is capitalized. However, in most cases, you want to be consistent with front-end developers, i.e., you want the first letter to be lowercase, as in http://localhost:5141/.stickers like this. The Core Web API provides the solution inJust add the following code to the file:
(options =>
{
= true;
= true;
});
Tip: Make Controller Methods Support Async Suffixes
In the StickersController controller, we use async/await for each API method, according to the C# programming specification, asynchronous methods should be suffixed with the word Async, but if this is done then theCreateAsync
This method returnsCreatedAtAction(nameof(GetByIdAsync), new { id }, sticker)
The following error is reported:
: No route matches the supplied values.
The solution is simple.file, call the
();
method when changing it to:
(options =>
{
= false; // Other code omitted...
// Other code omitted...
}).
At this point, the basic part of StickersController is complete, start the whole project, open the Swagger page, you can see the several APIs we have developed. now you can call these methods directly in the Swagger page to experience these RESTful APIs provided by our Sticker microservice:
summarize
This article describes the basic implementation of the Sticker microservice in our case, including the data access part and the design and implementation of the Sticker RESTful API, although at the moment we're just using anInMemoryDataAccessor
to emulate back-end data storage, but the basic functionality of the Sticker microservice is already there. However, in order to achieve cloud native, we also need to add some non-business related things to this Sticker microservice, such as: adding logging features to support runtime problem tracking and diagnosis; adding health check mechanism (health check) to support service state monitoring and running instance scheduling, in addition to the RESTful API Swagger documentation of the In addition, there are RESTful API Swagger documentation improvement, the use of version numbers and Git Hash to support continuous integration and continuous deployment and so on, these elements seem quite simple, but also need to spend a certain amount of time and effort to follow the standard best practices. I'll use a separate section to cover these once we're actually done with Sticker microservices.
In addition, the functionality of the Core Web API is not limited to what we have used so far. Since we are not focusing on learning the Core Web API itself, we will only cover the functionality we have used here, and readers who are interested in the entire knowledge structure of the Core Web API system are advised to read theOfficial Microsoft Documentation。
In the next talk I'll cover how to use PostgreSQL as the database for Sticker microservices, and from there I'll gradually introduce container technology.
source code (computing)
The source code for this chapter can be found here:/daxnet/stickers/tree/chapter_2/
Feel free to leave a comment with any questions about the code.