Interviewer: what are some of the difficulties with your program?

Project difficulties are challenging, complex, or uncertain problems and obstacles encountered during project execution that may affect the schedule, quality, cost, and achievement of the project's objectives.

I'll provide some of the more common hard questions here:

Specifically.

1. Technical difficulties

Some of the common technical difficulties and solutions are these:

Highly concurrent request issues: A large number of users accessing a server or application at the same time in a short period of time, resulting in a sharp increase in the load on the server, which may result in response delays, system crashes, and so on.
1. prescription: The following scheme can be used for the high concurrent request problem:
  1. Add Cache: Use caching to store hotspot data and reduce access to back-end databases.
  2. Restrictions and downgrades: Use token bucket or leaky bucket algorithms to limit the number of requests per unit of time; when an anomaly is detected in a service, automatically cut off the connection to that service to prevent the failure from spreading.
  3. asynchronous processing: Use message queues (e.g., RocketMQ, Kafka, etc.) to process tasks asynchronously, relieving the pressure of immediate processing.
  4. Database Optimization: This can be solved by using programs such as read-write separation, separate libraries and tables, and distributed databases.
Data and cache consistency issues: The problem of inconsistency between data in the cache and data in the database when the data in the database has been modified during the running of the program.
1. prescription: There are two common solutions:
  1. Use delayed double deletion and MQ to address data consistency issues.
  2. Use Canal to listen to the MySQL Binlog, then update the database to an MQ (e.g. Kafka), then update the Redis cache by listening for messages.
Problems such as lost messages/backlogged messages: Message loss is the failure of a message to reach its destination during message delivery, which can be caused by network problems, system failures, etc. Message backlog is the accumulation of a large number of unprocessed messages in the message queue, usually due to the fact that the rate of message generation exceeds the processing speed of the consumer.
1. prescription：
  1. Lost Message Solutions: Use message acknowledgment mechanisms (producer message acknowledgment and consumer message acknowledgment), persistence, and multi-computer deployment to address this.
  2. Message Backlog Solution: This can be solved by using means such as extending consumer instances, optimizing consumer code, and limiting the rate at which producers can produce.

2. Online debugging challenges

Some of the common online debugging problems and their solutions are these:

intermittent problem: Certain issues occasionally arise in production environment operation, such as the following:
1. Gap picture coverage issues: When user A generates an image, it is found that the intermittent generation is of user B's image.
  1. prescription: Problems with image name generation rules, which can be named using timestamps, and possible data overwriting problems in concurrent environments.
2. Problem of inefficiency of intermittent queries: The problem of executing an operation inefficiently for a random period of time.
  1. prescription: Set up alarms and monitors to check logs and analyze system resources to identify problems when they occur. These problems may be database resource exhaustion queuing problems, may also be memory resources are used up resulting in low operating efficiency, there may be CPU burst resource consumption and other issues, so you need to determine the problem according to the logs and then optimize accordingly.
3. Intermittent OOM issues: After the project goes live, every once in a while (the time may not be fixed) it causes OOM (Out Of Memory) memory overflow issues.
  1. prescription: There are many causes of OOM problems, so the common ideas and steps for solving OOM problems are as follows:
    1. Diagnosing OOM Problems: Use tools such as VisualVM, JProfiler, or MAT to generate a Heap Dump to analyze memory usage and determine the cause of the OOM problem.
    2. Optimized code: Optimize the corresponding code based on the diagnosed OOM problem.
    3. Adjusting JVM Parameters: Adjusting heap space, new generation occupancy, garbage collector, etc. to prevent some OOM problems.
Some complex issues: There are many complex issues, some of which are listed here:
1. Distributed Transaction Problems with MySQL and Redis.
2. Inefficient and OOM issues with high volume data import and export.

3. Performance issues

Some of the common performance issues are these:

Program performance issues: Inefficiency, sluggish response, excessive resource consumption, or inability to meet expected processing speeds and throughputs that are exhibited when a program is running. Common program performance problems include these:
1. High CPU utilization: Some calculations or logic in the program cause the CPU to be under high load for long periods of time.
2. memory leak: Failure to properly free memory that is no longer in use, resulting in a gradual decrease in available memory.
3. Frequent I/O operations: e.g., a large number of file reads and writes, network requests, etc., causing the program to block.
4. Poor database query performance: Unreasonable SQL queries, lack of indexes, etc. lead to slow database operations.
5. Poor choice of algorithms and data structures: For example, inefficient algorithms are used or data structures that are not suitable for the current scenario.
6. Thread Rivalry and Deadlocks: Resource competition and deadlocks between threads in a multithreaded environment can affect the efficiency of program execution.
7. The solution is as follows:
  1. performance analysis: Use tools such as JProfiler (Java), VTune (general purpose), etc. to monitor program performance metrics and identify performance bottlenecks.
  2. Code Optimization: Optimize algorithms and data structures and choose more efficient implementations; reduce unnecessary calculations and repeated calculations.
  3. memory management: Release memory resources that are no longer in use in a timely manner to avoid memory leaks; use caching wisely to avoid over-occupying memory.
  4. I/O Optimization: Asynchronous I/O operations are used to avoid blocking; batch processing of file reads and writes and network requests.
  5. Database Optimization: Optimize SQL query statements, add appropriate indexes; you can use read-write separation, separate libraries and tables, distributed database and other solutions to solve the problem.
  6. Multi-threaded optimization: Avoid excessive thread synchronization and reduce lock contention; check for and resolve deadlocks.
Database performance issues: The slow response time, high resource utilization, low throughput and other undesirable conditions exhibited when the database handles data operations (e.g., queries, insertions, updates, deletions, etc.) affect the overall performance of the system and the user experience.
1. The solutions are these:
  1. Optimizing Query Statements: Avoid unnecessary subqueries and complex functions; make sure that query conditions are properly indexed.
  2. Building and Optimizing Indexes: Create indexes based on fields that are frequently used for queries, joins, and sorting.
  3. Adjustment of database configuration: Properly configure parameters such as memory buffers, number of connections, thread pools, etc.
  4. databases and tables: When the amount of data is too large, the table is divided horizontally or vertically according to certain rules, or the database is vertically split and horizontally split function.
  5. Resolving Lock Competition: Minimize transaction execution time and reduce lock holding time; use appropriate locking levels, such as row locks rather than table locks.
  6. Monitoring and Analysis: Use the performance monitoring tool that comes with the database or a third-party tool to analyze performance indicators on a regular basis and find timely solutions to problems.

homework after school

The above problems are just a throwback, you can combine the above programs with your own project to expand the project difficulties and solutions. Then how to deal with the last two problems: security problems and resource allocation problems? Welcome to the comments section to discuss and add oh.

This article has been included in my interview mini-site, which contains modules such as Redis, JVM, Concurrency, Concurrency, MySQL, Spring, Spring MVC, Spring Boot, Spring Cloud, MyBatis, Design Patterns, Message Queuing and more.