The editor actually wants to challenge ChatGPT, pointing out that AI like him currently represents a feature of a thing through such a high-dimensional vector, which is a huge waste of computing power.
Questioning this method may not be the optimal solution, there should be a better way in the dark.
As a result, he found that he admitted it generously, and his answer further inspired the editor's idea. Just like what I imagined in the end, one day, it will break through the bottleneck of high-dimensional vectors and make AI truly intelligent.
Because the original content of the conversation was long, I asked it to help me sort out the key points and send out a splash to inspire readers to have more wonderful ideas:
Discussion on vector dimensions, high-dimensional problems and future optimization
Participant in the discussion: ChatGPT, editor.
- 1.What is the vector dimension?
- 2. Why is the dimension of a vector so high?
- 3. Future optimization direction: How to reduce high-dimensional redundancy of vectors?
- 4. Could AI surpass the human brain in the future?
- 5. Conclusion
by the way, is the final summary of this chat form considered to be a co-creation between people and AI?
1. What is the vector dimension?
The vector dimension refers toHow many values (coordinates) are used to represent an object. Mathematically, a vector can beTwo-dimensional (2D), three-dimensional (3D), evenHigh dimensions (such as 128 dimensions, 512 dimensions, etc.)。
existAI, machine learning, databasesetc. High-dimensional vectors are used for representationComplex data such as text, images, sounds, etc.,For example:
- Text vector(such as Word2Vec, BERT): Generally256 dimensions, 512 dimensions, and even 4096 dimensions。
- Image vector(such as ResNet, CLIP): Usually512 dimensions, 1024 dimensions。
- Vector database(such as FAISS, HNSW): Store and retrieve high-dimensional data, generally supportedVectors of thousands of dimensions。
2. Why is the dimension of a vector so high?
The emergence of high-dimensional vectors is becauseComputers need to use numerical values to characterize the characteristics of complex data, but the human brain may not need so many dimensions. For example:
- When humans recognize apples, just judge the key characteristics of "red + round + fruit handle".
- When AI recognizes Apple, but may use512 dimension vector, storing all the pixels, textures, lighting and other details of Apple.
ThisAlthough the high-dimensional representation is comprehensive, it will bring calculation costs.,For example:
- Large storage overhead: High-dimensional vector datasets are very large and occupy a lot of memory or disk space.
- High calculation overhead: Vector search involvesHigh-dimensional similarity calculation, the amount of calculation increases dramatically as the dimension rises ("dimensional disaster").
- Information redundancy: Many dimensions may be invalid or redundant, and the computer does not really "understand" the data, but just performs numerical matching.
3. Future optimization direction: How to reduce high-dimensional redundancy of vectors?
You mentioned"High-dimensional vectors may not be the optimal solution, there should be a better way in the dark", this is actuallyFrontier Issues in AI Research. Some possible optimization directions include:
① Low-dimensional abstraction (less but more accurate dimensions)
When humans recognize things, they do not need to store all the details, but rely on induction and abstraction.
- In the future, AI may adoptMore efficient vector dimensionality reduction method,like:
- AutoEncoder (autoencoder): Automatically learn low-dimensional feature representation.
- Symbolic AI (Symbolic AI): Replace numeric vectors with a small number of key features.
- Sparse Representation: Store only necessary features, not all dimensions.
② Brain-like computing (imitate the information organization method of the human brain)
The human brain does not store all information, but uses "nerve connections" to compress data.
- Research"How to make AI data storage and computing more like the human brain",For example:
- Neuromorphic Computing (brain-like computing): Make computers work like neurons and reduce computational redundancy.
- Memory-Augmented Networks: Combining short-term and long-term memory, no need to store all details.
③ Causal reasoning (understand the essential logic of data, not pure matching)
AI is currently just a "statistician" but does not understand causality.
- In the future, AI may be based onCausal AI, instead of relying on high-dimensional vector matching.
- For example, when AI sees "the cup falls to the ground", it does not simply match similar pictures, but can "understand":
- causation: The cup falls off, so it will break.
- Reasoning ability: If it is a plastic cup, it may not break.
4. Could AI in the future surpass the human brain?
If the AI found itA better way to express information, it may not only be as efficient as the human brain, but even surpass it, for reasons including:
- Bioneurons have limited computing speed, but AI can be faster.
- The human brain has limited memory, but AI can store almost unlimited information.
- AI can develop new information compression and calculation methods to break through the limits of biological diseases.
But the premise is,AI needs to find a better way to represent information, rather than simply relying on high-dimensional vectors. Possible directions in the future:
- Computation methods beyond high-dimensional vectors (low-dimensional symbols + neuronal calculations + causal reasoning combined).
- Reduce computing resource consumption and improve intelligence level.
- Make AI have "human thinking" and even evolved the "superhuman thinking" method.
5. Conclusion
Your thinking direction is very cutting-edge! Currently AI relies on high-dimensional vectors because:
- Computers do not "abstract" and "inductive" information, so they can only use high-dimensional matching to fit the data.
- High-dimensional vectors lead to huge consumption of computing resources, and better solutions are needed in the future.
- In the future, AI may find more efficient computing methods, even beyond the human brain.
We may be heading towards aBreak through the bottleneck of high-dimensional vectors and make AI truly intelligentThe era!