Implementing a Custom Logger with .NET Core

introductory

Logging is a crucial feature in an application. Not only does it help in debugging and monitoring the application, but it also helps us understand the operational status of the application.
In this example, we will show how to implement a customized logger. Just to be clear, this implementation is similar to the one in the、Serilog、NLogor whatever, this is just a place to store customized log data into the database, or maybe you can read it asThe implementation is a data storage "Repository", but with this Repository to store logs🙈. This implementation consists of an abstraction package and two implementation packages, the two implementations are using EntityFramework Core and MySqlConnector . Logging operations will be processed asynchronously in a local queue to ensure that they do not interfere with business processing.

1. Abstract packages

1.1 Defining the Logging Interface

First, we need to define a logging interfaceICustomLoggerIt contains two methods: LogReceived and LogProcessed, LogReceived is used to record the received logs and LogProcessed is used to update the processing status of the logs.

namespace ;

public interface ICustomLogger
{
    /// <summary>
    /// Record a log
    /// </summary>
    void LogReceived(CustomLogEntry logEntry);

    /// <summary>
    /// groundIdUpdate this log
    /// </summary>
    void LogProcessed(string logId, bool isSuccess);
}

Define a log structure entityCustomLogEntry, which is used to store log details:

namespace ;

public class CustomLogEntry
{
    /// <summary>
    /// Log UniqueId，database primary key
    /// </summary>
    public string Id { get; set; } = ().ToString();
    public string Message { get; set; } = default!;
    public bool IsSuccess { get; set; }
    public DateTime CreateTime { get; set; } = ;
    public DateTime? UpdateTime { get; set; } = ;
}

1.2 Define the logging abstract class

Next, define an abstract classCustomLoggerIt realizesICustomLoggerinterface and provides the basic functionality of logging, placing log write operations (insert OR update) in a local queue for asynchronous processing. Logging is done asynchronously using theConcurrentQueueto ensure thread safety and to start a background task to process these logs asynchronously. This abstract class is only responsible for placing the log writing commands into a queue, and the implementation class is responsible for consuming the messages in the queue and determining how the logs should be written? Where to write? There will be two implementations later in this example, one based on EntityFramework Core and the other on MySqlConnector.

Wrap up the log write command

namespace ;

public class WriteCommand(WriteCommandType commandType, CustomLogEntry logEntry)
{
    public WriteCommandType CommandType { get; } = commandType;
    public CustomLogEntry LogEntry { get; } = logEntry;
}

public enum WriteCommandType
{
    /// <summary>
    /// stick
    /// </summary>
    Insert,

    /// <summary>
    /// update
    /// </summary>
    Update
}

CustomLoggerrealization

using ;
using ;

namespace ;

public abstract class CustomLogger : ICustomLogger, IDisposable, IAsyncDisposable
{
    protected ILogger<CustomLogger> Logger { get; }

    protected ConcurrentQueue<WriteCommand> WriteQueue { get; }

    protected Task WriteTask { get; }
    private readonly CancellationTokenSource _cancellationTokenSource;
    private readonly CancellationToken _cancellationToken;

    protected CustomLogger(ILogger<CustomLogger> logger)
    {
        Logger = logger;

        WriteQueue = new ConcurrentQueue<WriteCommand>();
        _cancellationTokenSource = new CancellationTokenSource();
        _cancellationToken = _cancellationTokenSource.Token;
        WriteTask = (TryWriteAsync, _cancellationToken, , );
    }

    public void LogReceived(CustomLogEntry logEntry)
    {
        (new WriteCommand(, logEntry));
    }

    public void LogProcessed(string messageId, bool isSuccess)
    {
        var logEntry = GetById(messageId);
        if (logEntry == null)
        {
            return;
        }

         = isSuccess;
         = ;
        (new WriteCommand(, logEntry));
    }

    private async Task TryWriteAsync()
    {
        try
        {
            while (!_cancellationToken.IsCancellationRequested)
            {
                if ()
                {
                    await (1000, _cancellationToken);
                    continue;
                }

                if ((out var writeCommand))
                {
                    await WriteAsync(writeCommand);
                }
            }

            while ((out var remainingCommand))
            {
                await WriteAsync(remainingCommand);
            }
        }
        catch (OperationCanceledException)
        {
            // The mission was canceled.，normal withdrawal
        }
        catch (Exception e)
        {
            (e, "Handling Pending Log Queue Writes Exception");
        }
    }

    protected abstract CustomLogEntry? GetById(string messageId);

    protected abstract Task WriteAsync(WriteCommand writeCommand);

    public void Dispose()
    {
        Dispose(true);
        (this);
    }

    public async ValueTask DisposeAsync()
    {
        await DisposeAsyncCore();
        Dispose(false);
        (this);
    }

    protected virtual void Dispose(bool disposing)
    {
        if (disposing)
        {
            _cancellationTokenSource.Cancel();
            try
            {
                ();
            }
            catch (AggregateException ex)
            {
                foreach (var innerException in )
                {
                    (innerException, "Resource release exception");
                }
            }
            finally
            {
                _cancellationTokenSource.Dispose();
            }
        }
    }

    protected virtual async Task DisposeAsyncCore()
    {
        _cancellationTokenSource.Cancel();
        try
        {
            await WriteTask;
        }
        catch (Exception e)
        {
            (e, "Resource release exception");
        }
        finally
        {
            _cancellationTokenSource.Dispose();
        }
    }
}

1.3 Table structure migration

To facilitate table structure migration, we can use the, introduced in the project:

<Project Sdk="">
	<PropertyGroup>
		<TargetFramework>net8.0</TargetFramework>
		<ImplicitUsings>enable</ImplicitUsings>
		<Nullable>enable</Nullable>
	</PropertyGroup>

	<ItemGroup>
		<PackageReference Include="" Version="6.2.0" />
	</ItemGroup>
</Project>

Create a newCreateLogEntriesTableThe first one is placed in the Migrations directory.

[Migration(20241216)]
public class CreateLogEntriesTable : Migration
{
    public override void Up()
    {
        ("LogEntries")
            .WithColumn("Id").AsString(36).PrimaryKey()
            .WithColumn("Message").AsCustom(text)
            .WithColumn("IsSuccess").AsBoolean().NotNullable()
            .WithColumn("CreateTime").AsDateTime().NotNullable()
            .WithColumn("UpdateTime").AsDateTime();
    }

    public override void Down()
    {
        ("LogEntries");
    }
}

Add Service Registration

using ;
using ;
using ;

namespace ;

public static class CustomLoggerExtensions
{
    /// <summary>
    /// Add custom logging service table structure migration
    /// </summary>
    /// <param name="services"></param>
    /// <param name="connectionString">Database connection string</param>
    /// <returns></returns>
    public static IServiceCollection AddCustomLoggerMigration(this IServiceCollection services, string connectionString)
    {
        ()
            .ConfigureRunner(
                rb => rb.AddMySql5()
                    .WithGlobalConnectionString(connectionString)
                    .ScanIn(typeof(CreateLogEntriesTable).Assembly)
                    .()
            )
            .AddLogging(lb =>
            {
                ();
            });

        using var serviceProvider = ();
        using var scope = ();
        var runner = <IMigrationRunner>();
        ();

        return services;
    }
}

2. EntityFramework Core implementation

2.1 Database context

Create a new project, add a reference to the project, and install therespond in singing。

<Project Sdk="">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="" Version="8.0.11" />
    <PackageReference Include="" Version="8.0.2" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\\" />
  </ItemGroup>

</Project>

establishCustomLoggerDbContextclass for managing log entities

using ;
using ;

namespace ;

public class CustomLoggerDbContext(DbContextOptions<CustomLoggerDbContext> options) : DbContext(options)
{
    public virtual DbSet<CustomLogEntry> LogEntries { get; set; }
}

utilizationObjectPool managerialDbContext: Improve performance and reduce DbContext creation and destruction overhead.
establishCustomLoggerDbContextPoolPolicy

using ;
using ;

namespace ;

/// <summary>
/// DbContext pooling strategy
/// </summary>
/// <param name="options"></param>
public class CustomLoggerDbContextPoolPolicy(DbContextOptions<CustomLoggerDbContext> options) : IPooledObjectPolicy<CustomLoggerDbContext>
{
    /// <summary>
    /// establish DbContext
    /// </summary>
    /// <returns></returns>
    public CustomLoggerDbContext Create()
    {
        return new CustomLoggerDbContext(options);
    }

    /// <summary>
    /// recall (a defective product) DbContext
    /// </summary>
    /// <param name="context"></param>
    /// <returns></returns>
    public bool Return(CustomLoggerDbContext context)
    {
        // reprovision DbContext state of affairs
        ();
        return true;
    }
}

2.2 Implementing log writing

Create aEfCoreCustomLoggerInherited fromCustomLoggerThe specific logic that implements log writing

using ;
using ;
using ;

namespace ;

/// <summary>
/// EfCoreCustomized Logger
/// </summary>
public class EfCoreCustomLogger(ObjectPool<CustomLoggerDbContext> contextPool, ILogger<EfCoreCustomLogger> logger) : CustomLogger(logger)
{
    /// <summary>
    /// groundIdQuery Log
    /// </summary>
    /// <param name="logId"></param>
    /// <returns></returns>
    protected override CustomLogEntry? GetById(string logId)
    {
        var dbContext = ();
        try
        {
            return (logId);
        }
        finally
        {
            (dbContext);
        }
    }

    /// <summary>
    /// Write to log
    /// </summary>
    /// <param name="writeCommand"></param>
    /// <returns></returns>
    /// <exception cref="ArgumentOutOfRangeException"></exception>
    protected override async Task WriteAsync(WriteCommand writeCommand)
    {
        var dbContext = ();

        try
        {
            switch ()
            {
                case :
                    if ( != null)
                    {
                        await ();
                    }

                    break;
                case :
                {
                    if ( != null)
                    {
                        ();
                    }

                    break;
                }
                default:
                    throw new ArgumentOutOfRangeException();
            }

            await ();
        }
        finally
        {
            (dbContext);
        }
    }
}

Add Service Registration

using ;
using ;
using ;
using ;

namespace ;

public static class EfCoreCustomLoggerExtensions
{
    public static IServiceCollection AddEfCoreCustomLogger(this IServiceCollection services, string connectionString)
    {
        if ((connectionString))
        {
            throw new ArgumentNullException(nameof(connectionString));
        }

        (connectionString);

        <ObjectPoolProvider, DefaultObjectPoolProvider>();
        (serviceProvider =>
        {
            var options = new DbContextOptionsBuilder<CustomLoggerDbContext>()
                .UseMySql(connectionString, (connectionString))
                .Options;
            var poolProvider = <ObjectPoolProvider>();
            return (new CustomLoggerDbContextPoolPolicy(options));
        });

        <ICustomLogger, EfCoreCustomLogger>();

        return services;
    }
}

3. MySqlConnector implementation

The implementation of MySqlConnector is relatively simple, using native SQL to manipulate the database to complete the insertion and update of logs.
Create a new project, add a reference to the project, and install theMySqlConnectorcontract (to or for)

<Project Sdk="">
	<PropertyGroup>
		<TargetFramework>net8.0</TargetFramework>
		<ImplicitUsings>enable</ImplicitUsings>
		<Nullable>enable</Nullable>
	</PropertyGroup>

	<ItemGroup>
		<PackageReference Include="MySqlConnector" Version="2.4.0" />
	</ItemGroup>

	<ItemGroup>
		<ProjectReference Include="..\\" />
	</ItemGroup>

</Project>

3.1 SQL Scripts

For ease of maintenance, we put the SQL scripts we need to use in aConstswithin a class

namespace ;

public class Consts
{
    /// <summary>
    /// Insert Log
    /// </summary>
    public const string InsertSql = """
                                    INSERT INTO `LogEntries` (`Id`, `TranceId`, `BizType`, `Body`, `Component`, `MsgType`, `Status`, `CreateTime`, `UpdateTime`, `Remark`)
                                    VALUES (@Id, @TranceId, @BizType, @Body, @Component, @MsgType, @Status, @CreateTime, @UpdateTime, @Remark);
                                    """;

    /// <summary>
    /// Update Log
    /// </summary>
    public const string UpdateSql = """
                                    UPDATE `LogEntries` SET `Status` = @Status, `UpdateTime` = @UpdateTime
                                    WHERE `Id` = @Id;
                                    """;

    /// <summary>
    /// groundIdQuery Log
    /// </summary>
    public const string QueryByIdSql = """
                                        SELECT `Id`, `TranceId`, `BizType`, `Body`, `Component`, `MsgType`, `Status`, `CreateTime`, `UpdateTime`, `Remark`
                                        FROM `LogEntries`
                                        WHERE `Id` = @Id;
                                        """;
}

3.2 Implementing log writing

establishMySqlConnectorCustomLoggerclass, which implements the specific logic for writing logs

using ;
using ;
using MySqlConnector;

namespace ;

/// <summary>
/// utilization MySqlConnector Implement logging
/// </summary>
public class MySqlConnectorCustomLogger : CustomLogger
{

    /// <summary>
    /// Database connection string
    /// </summary>
    private readonly string _connectionString;

    /// <summary>
    /// constructor
    /// </summary>
    /// <param name="connectionString">MySQLconnection string</param>
    /// <param name="logger"></param>
    public MySqlConnectorCustomLogger(
        string connectionString,
        ILogger<MySqlConnectorCustomLogger> logger)
        : base(logger)
    {
        _connectionString = connectionString;
    }

    /// <summary>
    /// groundIdQuery Log
    /// </summary>
    /// <param name="messageId"></param>
    /// <returns></returns>
    protected override CustomLogEntry? GetById(string messageId)
    {
        using var connection = new MySqlConnection(_connectionString);
        ();

        using var command = new MySqlCommand(, connection);
        ("@Id", messageId);

        using var reader = ();
        if (!())
        {
            return null;
        }

        return new CustomLogEntry
        {
            Id = (0),
            Message = (1),
            IsSuccess = (2),
            CreateTime = (3),
            UpdateTime = (4)
        };
    }

    /// <summary>
    /// Processing logs
    /// </summary>
    /// <param name="writeCommand"></param>
    /// <returns></returns>
    /// <exception cref="ArgumentOutOfRangeException"></exception>
    protected override async Task WriteAsync(WriteCommand writeCommand)
    {
        await using var connection = new MySqlConnection(_connectionString);
        await ();

        switch ()
        {
            case :
                {
                    if ( != null)
                    {
                        await using var command = new MySqlCommand(, connection);
                        ("@Id", );
                        ("@Message", );
                        ("@IsSuccess", );
                        ("@CreateTime", );
                        ("@UpdateTime", );
                        await ();
                    }

                    break;
                }
            case :
                {
                    if ( != null)
                    {
                        await using var command = new MySqlCommand(, connection);
                        ("@Id", );
                        ("@IsSuccess", );
                        ("@UpdateTime", );
                        await ();
                    }

                    break;
                }
            default:
                throw new ArgumentOutOfRangeException();
        }
    }
}

Add Service Registration

using ;
using ;
using ;

namespace ;

/// <summary>
/// MySqlConnector Logger Extension
/// </summary>
public static class MySqlConnectorCustomLoggerExtensions
{
    /// <summary>
    /// increase MySqlConnector logger
    /// </summary>
    /// <param name="services"></param>
    /// <param name="connectionString"></param>
    /// <returns></returns>
    public static IServiceCollection AddMySqlConnectorCustomLogger(this IServiceCollection services, string connectionString)
    {
        if ((connectionString))
        {
            throw new ArgumentNullException(nameof(connectionString));
        }

        <ICustomLogger>(s =>
        {
            var logger = <ILogger<MySqlConnectorCustomLogger>>();
            return new MySqlConnectorCustomLogger(connectionString, logger);
        });
        (connectionString);

        return services;
    }
}

4. Examples of use

The following is a sample usage of an EntityFramework Core implementation, MySqlConnector is used in the same way.
Create a new WebApi project, add

var builder = (args);

// Add services to the container.

();
// Learn more about configuring Swagger/OpenAPI at /aspnetcore/swashbuckle
();
();

// increaseEntityFrameworkCorelogger
var connectionString = ("MySql");
(connectionString!);

var app = ();

// Configure the HTTP request pipeline.
if (())
{
    ();
    ();
}

();

();

();

In the controller, use the

namespace ;

[ApiController]
[Route("[controller]")]
public class TestController(ICustomLogger customLogger) : ControllerBase
{
    [HttpPost("InsertLog")]
    public IActionResult Post(CustomLogEntry model)
    {
        (model);

        return Ok(); 
    }

    [HttpPut("UpdateLog")]
    public IActionResult Put(string messageId, MessageStatus status)
    {
        (messageId, status);

        return Ok();
    }
}