The four main characteristics of a transaction
Atomicity, Isolation, Persistence, Consistency
Transaction isolation levels and phenomena
- Read uncommitted: may produce dirty reads, reading uncommitted data
- Read committed: may produce non-repeatable read problems, A transaction read to the B transaction has been committed data, resulting in two read data inconsistencies
- Repeatable reads: may produce phantom read problem, A transaction query to B transaction inserted data, resulting in two query entries are inconsistent
- Serializable: highest level, forced transaction serialization execution, exclusive locking, slow concurrency efficiency
Difference between unrepeatable and phantom reads
Non-repeatable reads focus on modifications, with different data in the two queries
Phantom reads focus on additions and deletions, with different rows in the two queries
How Spring solves circular dependency problems
First, let's talk about what a circular dependency is: A object depends on B object, and B object depends on A object.
Spring's core idea of solving circular dependencies lies in early exposure, first create instantiation A, and save the lambda expression that instantiates A in the three-level cache in order to get an instance of A, and then go to instantiate B. You can fetch A when instantiating B, and then go to instantiate A later.
When I don't have circular dependencies and AOP, this three-level cache is not used in the follow-up.
Threads have several states
The 6 states are: New, Running, Blocking, Waiting, Timeout Waiting, Terminated
Difference between wait and sleep
- wait comes from the Object class and Sleep comes from the Thread class.
- wait releases the lock, sleep does not.
- WAIT must be executed in a synchronized code block, SLEEP can be executed at will.
- wait can catch exceptions, sleep must catch exceptions.
Difference between synchronized and lock locks
sync reentrant lock, non-interruptible, non-fair locks
lock reentrant lock, judgment lock, non-fair lock (settable)
What is a reentrant lock?
As long as you hold the lock, you don't need to contend for the lock when you re-enter the block, e.g., recursively, and you can keep going in and out without deadlocking.
Anomalies encountered at work,
Runtime Exceptions, IO Exceptions, Database Connection Exceptions, ClassNotfound Exceptions, OutOfMemoryError Exceptions, ConcurrentModification Concurrent Modification Exceptions
Difference between signals and mutex locks
Signal volume: can be held by multiple threads at a time, resource scheduling
Mutual exclusion locks: can only be held by one thread at a time, resource mutual exclusion
What is AQS
Abstract object synchronizer, based on CLH queue (bidirectional linked table), with volatile modification of the shared state (state), the thread through the CAS to change the state symbols, the success of the success of the acquisition of the lock, failure to enter the waiting queue, waiting to be woken up.
The three main methods of the Executors thread pool factory
newSingleThreadExecutor() single thread pool, newFixedThreadPool(n) creates a fixed-size thread pool, newCachedThreadPool() scalable thread pool
singleton model
Hungry Man style: direct object creation.
Lazy: object creation on use, double-detect locking and volatile keyword to solve concurrency issues.
CAS (compare-and-swap-optimistic locking) mechanism
Compares the current value in working memory with the value in main memory, if this value is expected, the operation is performed, otherwise it is not performed.
There is an ABA beaver problem:
This can be solved by using an increased version number, optimistic locking mechanism
JUC provides AtomicStampedReference, an atomic timestamp reference class, which solves the ABA problem
How to troubleshoot deadlock problems
Locate the process: jps -l
Stack trace: jstack process number
JVM Memory Model
- Heap: memory objects
- Stack: local variables, method exports, operands, dynamic links
- Method area (meta space): constants, static variables, class information
- Native method stack: native methods
- Program counter: the execution sequence number of the decompiled code
Introduction to JVM Parameters
-Xmx maximum heap memory
-Xms initial heap memory
JVM garbage collection mechanism
JVM: MINORGC will occur after the young generation memory is full, using reachability analysis algorithm, put the referenced object into the e0 area, for the referenced object will be minorgc garbage collection, every time minorgc occurs, the surviving object will be switched in the s0 and s1 area, when the s area is full and the sub-generation age 15, it will go into the old generation, the old generation is full, minorgc will happen. fullgc, if fullgc also can not release the object, and the new object has been generated and put into the old age will be memory overflow OOM.
JVM tuning: determine how many seconds the service business code will be executed * how many M objects are generated per second, and set the size of the young generation Eden (8:1:1) (the business code is just about finished executing, and minorgc occurs for recycling), and the rest of the memory is allocated to the old generation.
G1 collector (C++ language implementation), you can set the maximum stopping time through the parameter, reach this stopping time will occur gc, need to consider the business scenario, to avoid the object is too large or too old, most of the objects into the old age.
JVM Garbage Collection Algorithm
- Reference Counting Method: Program Counter
- Young Generation-Survivor Zone: Replication Algorithm
- The old days:
- Marker clearing (marking surviving objects and clearing unmarked objects creates memory fragmentation)
- Marking collation (mark surviving objects, move surviving objects to one end, clear unmarked objects, no memory fragmentation)
Why STW (stop while recalling) was designed
If not, fullgc and minorgc will delete non-garbage objects from the program by mistake.
Large objects (greater than 50%) in the s-area will be released directly to the old age.
minorgc and fullgc generate STW.
Why is there a division between younger and older generations in the heap?
A: If you don't divide it, the garbage objects keep piling up without being recycled, and when gc actually happens, it will take a super long time.
Why do young generations have s0 and s1 zones?
- If you don't have it, it will cause minorgc to take place and put the surviving object into the old age, resulting in fullgc and a long pause.
- If there is only one s-area, the copying algorithm creates memory fragmentation, and the eden and s0 areas copy each other, e.g., eden and s0 each have live objects, and eden puts the live objects into the s0 area, which results in memory discontinuities
- Why two districts, the optimal solution after trade-offs
JMM Memory Model
Mainly divided into the main memory (heap) and working memory (stack), working memory is the main memory and a copy of the main operations are: lock lock, unlock unlock, read read, load load, use use, assign assign, store store, write write
TCP handshake three times
The problem of the second handshake: the client's first request when the delay is sent to the server, the server successfully received the request (but in the client that this request has failed), the second response when the server responds to the client, the client believes that the request has failed, and will not receive this time to respond to the client, the server will always be deadlocked waiting (deadlocked) for the client to send the data message, resulting in resource consumption (if the client re-establishes the connection, a new channel will be established, the previous channel will always exist). In the case of three handshakes, the third handshake will close this connection channel if the server does not receive a response from the client within a period of time.
For the first handshake, the client sends a request to the server to establish a connection.
For the second handshake, the server responds to the client whether or not to confirm the establishment of the connection.
On the third handshake, the client sends an acknowledgement to the server to establish the connection.
Purpose of the triple handshake: to prevent a connection request message that has failed from suddenly being delivered to the service side
TCP waved four times.
As a result of the TCP bidirectional channels being independent of each other, the
- c sends "I want to close the connection" to s.
- s sends "I know" to c, and s enters a semi-closed state (can receive data, cannot send data).
- s sends "I'm disconnecting, confirm" to c to enter the final confirmation state.
- c sends "can disconnect" to s, s receives it and closes the connection, c waits for 2msl (the maximum time for a data message to go back and forth) and enters the closed state.
ArraysList Expansion Mechanism
Default initial capacity 10, expansion of 1.5 times the original capacity rounded up, expansion factor 1
Vector Vector Expansion Mechanism
Thread-safe sync locks
Default initial capacity 10, expansion 2 times the original capacity rounded up, expansion factor 1
HashMap expansion mechanism
Default initial capacity 16, expansion multiplier 2, expansion factor 0.75
Array length is greater than 64 and the length of the chain table is greater than or equal to 8 (every generated hash conflict, equal comparison, hash code is the same replacement, different inserted into the chain table) -> the chain table is transformed into a red-black tree, the length of the chain table is less than or equal to 6 from the red-black number to the chain table, jdk1.7 using the header insertion, jdk1.8 using the tail insertion.
HashMap data structure
The implementation of HashMap is based on a combination of an array and a chained table (or red-black tree) implementation. The array is initialized to a certain number of buckets, and each bucket can contain a linked table or a red-black tree. When a key-value pair is added to a HashMap using the put() method, a hash is first calculated for the key and the corresponding bucket is found. If the bucket is empty, the key-value pair is inserted into the bucket. If the bucket is not empty, iterates through all the key-value pairs in the bucket to find out if the same key already exists, and updates the value if it does, otherwise inserts the new key-value pair into the bucket.
ConcurrentHashMap data structure
1.7 The idea of ReentrantLock+Segment+HashEntry segmented locking is used to lock each Segment bucket bit.
1.8 Abandon the idea of segmented locks, closer to the hashmap, the use of synchronized + CAS + HashEntry + red-black tree, for each array element locking
HashMap addressing
put method actually has four parameters, the first parameter hash value is 32-bit, shifted left 16-bit and "high 16-bit and low 16-bit value obtained by dissimilarity" so that the value of the random more random, try to avoid hash collision, improve query efficiency.
Why HashMap is not thread-safe
Because when the put method is executed, the first thing to do is to get the index of the inserted position, if another thread enters at this time and gets the same index and inserts it, the current thread will overwrite the inserted data if it inserts it again, resulting in data inconsistencies
The three main characteristics of concurrency
Atomicity, visibility, ordering
Parental Delegation Mechanism
Class loaders : BootStrapClassLoader main loader , ExtClassLoader extension loader , AppClassLoader system class loader and thread context loader , you can customize the loader .
Biparental delegation: upward delegation to the top-level loader, downward lookup loading paths
Daemon Thread
Daemon threads are closed at the end, daemon threads cannot be created with the thread pool, daemon threads created by the thread pool are converted to user threads, and threads created in the daemon threads are created as daemon threads.
Thread Pool Execution Flow
More than the core thread (party A), it is put into the task queue (scheduling), the task can not be scheduled, put into the maximum thread (outsourcing), the maximum thread exceeds the idle time, the implementation of the rejection policy (dismissal)
keep timeout, there are threads in the max threads that are idle, after this time, the threads will be recalled.
Seven Parameters of Thread Pool
- corePoolSize The core thread size of the thread pool.
- maximumPoolSize The maximum number of threads in the thread pool.
- keepAliveTime idle thread alive time
- unit Idle thread survival time unit
- workQueue workQueue
- threadFactory thread factory
- handler Rejection Policy
- AbortPolicy: Discard the task and throw an exception.
- DiscardPolicy: Silently discard tasks without throwing exceptions.
- DiscardOldestPolicy: top tasks are discarded and new tasks are added to the queue
- CallerRunsPolicy: the task is performed by the calling thread and will not be dropped
What is spring?
- is a lightweight simplify the development of the framework , the container to manage the bean , the middleman of the various frameworks (integration of various frameworks)
- ioc containers are used to manage beans, using aop to solve the problem of oop code duplication
- springmvc is spring on the web framework of a solution , total front-end controller servlet view parser to generate views to the page .
- springboot is spring's rapid development package , equivalent to spring and springmvc integration , the convention is greater than the configuration .
Spring object life cycle
- Parse the class to get the BeanDefinition definition object.
- Inferring construction methods.
- Perform instantiation to get the object.
- Assigns a value to the @Autowired annotation property in the object.
- Callback to the Aware method (get container for accessing other beans in the container).
- Call the pre-initialization method.
- Call initialization.
- Call the initialized method.
- Put the singleton bean into the singleton pool.
- Use Bean.
- Destroy the bean.
Scope of bean
singleton single instance, prototype multiple instances, request a request a single instance, session a session a single instance, application a context a single instance, webSocket a connection a single instance
Is spring's singleton bean thread-safe?
No, the framework doesn't handle multithreading with double-detected locks and the visibility keyword, ThreadLocal or locking.
Principle of automatic assembly of spring
@Import imports the scanning META-INFO/ file, which configures the configuration classes in each starter, which are injected using @Bean, and each starter needs to adhere to springboot's conventions.
Explain the implementation plan
The main optimization looks at the type field and performs efficiency: avoid (All and index) All full table <index traverses the index tree <range retrieves a range of data <ref non-unique index scanning <eq_ref unique index scanning <system table has only one piece of data in it <const passes through the index once hit, matches one piece of data
What is the MVCC mechanism
A method of multi-version concurrency control, through the version chain (readView maintains the active transaction Id, sorted into an array from small to large, outside the array on the left side of the committed transactions can be accessed, outside the array on the right side of the uncommitted can not be accessed), to achieve multi-version, concurrent read and write, through the readView, the generation of different strategies, to achieve different isolation levels
Difference between RDB and AOP
RDB: Snapshot mode, through the configuration file save 600 3 settings, 600 seconds at least 3 key changes occur, generate a snapshot to replace the original snapshot.
AOF: file append mode, record the modification action to the file, append once a second by default, appendfsync: for configuration.
Difference: AOF efficiency is lower than RDB, AOF using append RDB using snapshots, RDB (if the machine is dead after the loss of data that did not generate a snapshot) without AOP (will lose a second of data, and will not affect the data that existed before) Data Security
Why redis is single-threaded
Only in the processing of network requests using a single thread, complete memory-based operations, non-blocking multiplexed IO multiplexing (using select, you can establish multiple socket connections to handle IO non-multi-threaded), to avoid context and multi-threaded switching to avoid locking problems caused by resource consumption
What are cache avalanche, cache hit, and cache penetration?
Cache avalanche: caches expire massively at the same time, expiration times are spaced as randomly as possible, cache warm-up-synchronize data to cache at startup.
Cache penetration: a large number of requests to access non-existent data at the same time, are going to mysql query, generally by the attack, can be resolved through the cache key-null ➕ expiration time, Bloom filters, code checksum.
Cache hit: refers to the hotspot data is invalidated, a large number of concurrently checking the same data leads to a crash, you can never expire through the hotspot data - modify the synchronization of the cache or after the expiration of the database access to add mutual exclusion locks
How to implement interface idempotency (multiple clicks cause data issues)
Use optimistic locking mechanism to add unique business flow numbers
Difference between HTTP and TCP
HTTP is located in the application layer. http is a short connection based on TCP.
The TCP protocol is located at the transport layer, and the TCP protocol supports both long and short connections.
redis basic data types
5 common data types, string,hash,list,set,zset
mysql slow query log analysis
Slow query logging can be turned on by command or configuration, mysql provides a tool mysqldumpslow can analyze the
Synchronized lock upgrade
Java Virtual Machine optimization of synchronized, most cases locks do not exist multi-threaded competition, so to the first site bias, once the case of multi-threaded competition, it will be upgraded from the bias lock to lightweight locks (spin locks), if the spin performs a certain number of times the CAS (Compare and Swap) has not been acquired locks, it will be inflated from the lightweight locks to heavyweight locks, which only allows one thread to enter the
Difference between process and thread
A process is the smallest unit of the system to allocate resources and a thread is the smallest unit of the system to schedule.
A program has at least one process, and a process has at least one thread.
The Java thread scheduling algorithm is a time-slice rotation approach.
pageHelper paging principle
Using Interceptors to Intercept and Rewrite SQL
The difference between mybatis # and $
(#) Pre-compile, the parameters will be added single quotes by default, use PreparedStatement's set method to assign values for the fetch scenario to prevent sql injection.
($)sql splicing, which can be used in dynamic table name/field scenarios and can lead to sql injection.
What is a reachability analysis algorithm
The GCRoot (reference to the virtual machine stack) starts searching downwards, and the path traveled by the search is called the reference chain; when the gc root of an object does not have any reference chain, it proves that the object is unavailable and unreachable.
What is a row lock and what is a table lock
A row lock is an exclusive lock that prevents other transactions from modifying the row.
Table locks are shared read locks, exclusive write locks
redis expiration policy and memory elimination mechanism
expiration strategy
Regular inerting
Periodic deletion-checks every 100 milliseconds to see if the key has expired, and deletes it if it does.
Inert deletion-checks for expiration when fetching the key and deletes it if it expires.
Memory elimination strategies:
If the periodic deletion did not delete the key, but also did not request the key for inert deletion, at this time the memory will be more and more large, you can configure the memory elimination policy in the configuration file, such as: from the key that has been set to expire out of the key that has not been used recently to eliminate the key
redis distributed locking implementation
setnx plus expiration time expire, but it is not executed synchronously, which may lead to deadlocks.
There are complex parameters in set, and you can use setnx and expire as a single command.
redis delayed queue implementation
Use zset and set a timestamp, then query all the delayed tasks, find out the delayed tasks that need to be processed, determine whether the timestamp is greater than the current system time, and process the message if it is greater than the current system time.
How to achieve data consistency
Delayed double deletion to achieve data consistency between mysql and redis
- Deleting redis cache data
- Update database data (here there may be other transactions that modify the data and successfully put it into the cache, causing the redis data to not be up-to-date)
- Delayed synchronization time (waiting for other update cache operations to complete)
- Deleting redis cache data
How to avoid double sending and double consumption of messages
Duplicate sends: MQ internally generates an inner-msg-id for each message sent by the producer as a basis for de-emphasis and idempotency
Repeat consumption: when consuming a message, a bizId is required in the message body
Common types of encryption algorithms
AES: Symmetric encryption - encryption and decryption are performed with the same key.
RSA: asymmetric encryption - public key (key) for encryption, private key (key) for decryption
Common linux commands
tail -f Real-time logging
tail -n 100 View the last 100 lines of the log
netstat -apn | grep 8080 Queries information on the specified port
ps -ef | grep pid Query information about a specified process
top View cpu usage
df -h Viewing Hard Disk Usage
free -h View memory usage
Differences between Innodb and Myisam storage engines
Innodb supports transactions, must have primary keys, table/row locks, MVCC mode, foreign keys
Myisam does not support transactions, foreign keys are not required, only table locks are supported, MVCC mode is not supported, foreign keys are not supported
Fundamentals of Mysql Master-Slave Synchronization
- The master library saves the modified statement to the binlog log
- The master library creates a thread to send the binlog log to the slave library
- Create an IO thread from the library to read the binlog logs
- Parsing statement execution in the binlog log from the library
Memory elimination strategy for redis
- noenviction: does not clear the data, just returns an error, which causes more memory to be wasted for most write commands (with the exception of the DEL command and a few other commands)
- allkeys-lru: selects the least recently used data from all datasets to be eliminated for use with new data
- volatile-lru: selects the least recently used data from a dataset that has been set to expire and eliminates it for new data
- allkeys-random: Arbitrarily select data elimination from all datasets for use with new data
- volatile-random: Arbitrarily select data from a dataset that has an expiration time set to be eliminated for use in new data
- volatile-ttl: selects data that will expire from a data set that has an expiration time set to be eliminated for use with new data
- volatile-lfu: eliminates the least frequently used key from all keys configured with an expiration time
- allkeys-lfu: eliminates the least frequently used key from all keys
Name a few scenarios where transactions fail
- Transaction annotations on non-public methods fail because the AOP proxy filters non-public methods.
- Method interop transactions in the same class will fail, reason: bypassing the AOP proxy object
- The database engine does not support transactions
- Exception eaten by catch
redis persistence mechanism
af: less data lost, slow recovery, write operations append files (logging), more disk occupied
rdb: more lost data, fast recovery, generation of snapshots at intervals, smaller files
Short url compression algorithm
- Use md5 encryption, split into four segments, take one at random
- Convert to hexadecimal (a-zA-Z0-9) using a puter, initially 0, incremented for new requests.
- Make a mapping between the short link server and the real path and save it to redis.
- Or use the LUR local cache to store the n most recently accessed mappings, update the time of the most recent mappings, and eliminate the most recently unvisited mappings
- Use 301 or 302 redirects to the real address depending on the mapping
- 301 permanent redirects:Unable to log hits
- 302 Temporary redirects: clicks can be recorded
Current Limiting, Fusing, Degradation
Meltdown - If a service is down, a friendly message is returned to the front-end, which will regularly check if the service is back up and slowly return to service.
Flow limiting - pages limit the number of requests per second QPS
Degradation - If a certain period of time, the amount of requests is too large, the server pressure is too great, you can turn off some unnecessary functions, to retain some core functions
springmvc execution flow
Front-end controller DispatcherServlet -> Handler mapper handlerMapping -> Handler adapter handlerAdapter -> ModelAndView -> View Resolver ViewReslove handles the ModelAndView ->. View view rendered to page
Database Tuning
When the table structure is built the field length is as wasteful as possible, because mysql underlying data storage for memory pages is 16kb, if a row of data is 16kb in size, then only one piece of data can be stored, it's horrible.
What is a memory page?
mysql can only read data from one memory page per IO.
Each node of a b+ number is a memory page of data.
Mysql Index Types
Clustered indexes (primary key indexes)
Auxiliary indexes (unique indexes, general indexes, combinatorial indexes)
Inverted indexes (full-text indexes)
Overriding indexes - no need to go back to the table to query
What problems might threadlocal cause?
Memory leaks, because the underlying use of threadlocalmap, the key is a soft reference object, the key may be recycled when gc, but the value is still in memory, so you need to explicitly clear the
Leftmost Matching Principle in Databases
For union indexes
- The joint index (a,b,c) is equivalent to creating the indexes: (a),(a,b),(a,b,c)
- mysql internal optimizer that automatically sorts where conditions and matches combined indexes
Logic for storing b+trees in databases
The root node of the b+ tree does not store data only keys, the leaf nodes store (aggregated indexes store data, non-aggregated indexes store primary keys and indexed columns), the individual pages of the tree are connected to each other (page16kb) by a bi-directional chained table, while the leaf node data is connected by a uni-directional chained table
AOP notification type
Pre-Notification, Post-Notification, Surround Notification, Exception Notification, Final Notification
What is idempotent?
The interface is only allowed to be called once and the effect on the data will only be triggered once
Interface idempotency (multiple clicks lead to data problems), multiple calls to the interface and one call with equal results
How does RabbitMQ work?
1. Producer
- Creating a switch, specifying the switch type
- Creating a Queue
- The queue is bound to a switch and can specify a routingKey.
- Sends messages to the switch and can specify routingKey
1. consumer
- Creating a queue
- Listener queue
What is the difference and role of each mode of RabbitMQ switch?
2. fanout, broadcast mode, sends messages to all bound queues
3. direct, route mode, sends a message to the queue with the specified routingKey.
4. topics, topic mode, sends a message to the queue with the specified routingKey (wildcards can be used for bindings).
5. headers and direct switches are identical, but the performance is much worse and almost unused
RabbitMQ Workflow
- ConnectionFactory --> Connection --> Channel --> Create Switch --> Create Queue --> Queue Bind Switch --> Send Message/Receive Message
- Send directly to queue: default switch is used on the bottom layer
Troubleshooting Ideas for Common RabbitMQ Failures
RabbitMQ Server Down Messages Lost
Message queues are persisted to the hard disk by default and do not fear reboots