Hello everyone, I am open source multidimensional table projectTeable 's founder, Gabe Chan.
As the earliest leader of Flying Book Multi-Dimensional Forms, I have participated in and witnessed the whole process of this product from 0 to 1. This experience also helped me understand the real needs of enterprises in data collaboration.
in order toAirtable cap (a poem)Notion For the representative of the spreadsheet database, showing an exciting direction: so that business people who do not know how to code can also build their own needs of the business system. However, as we go deeper, we find that all existing products have core contradictions that are difficult to break through: performance bottlenecks, data security and open capabilities.
Teable: Redefining Multidimensional Forms
Unlike other products on the market, theTeable The core concept is "database parity". We have observed that users expect more than just a more powerful spreadsheet, they expect multidimensional forms to actually carry core business data and applications.
Teable's goal is toOpen up specialized database capabilities to all through a no-code interface. We want to turn specialized databases that were meant for developers into no-code tools that can be used by people who don't know how to code. Databases shouldn't just be a tool for technicians, but a tool to support any team's business applications.
This led us to start the Teable project. Our thinking was straightforward:
- First, ensure complete control of data through open source
- Secondly, support for millions of data is as natural as breathing
- Finally, keep the database as easy to use as a table
This allows business people from all industries to easily build any management system like customer management, project tracking, order management, industry customized data systems, etc. directly on top of the database, and also easily integrate with existing data systems and tools to build further automated data processing, data insights, and third-party applications on top of the built applications.
The choice of open source is the result of much deliberation on our part. We believe that professional database capabilities should be at your fingertips, and the control of data should be returned to the users. Luckily, this idea has been widely recognized: in the 8 months after open source, Teable has gained 13,000 Github stars with almost no promotion, and has been on top of the Github trending for many times. it is now used by users in more than 100 countries and regions, and has been selected for the Runa 2024Q2 list of the most promising open source projects.
When Developers Meet O&M Challenges
We're a bunch of programmers and don't have dedicated Ops engineers.
During the development of the project, our team faced a number of technical infrastructure challenges. While we were very passionate about solving technical problems, it was clear that we didn't want to waste our valuable time dealing with O&M issues.
The manual deployment dilemma
At the beginning, we adopted the simplest solution - buying a server and deploying Docker containers by ourselves. We thought that since we don't have many users, buying a server and starting a few containers with Docker should be enough. But soon enough, problems ensued:
- Management of middleware such as databases, object stores, Redis, etc.
- SSL Certificate Renewal and CICD Process Maintenance
- System stability issues (e.g., lack of memory)
- Various technical details when expanding capacity
We waste a lot of time on things that have nothing to do with business.For example, each middleware has to be bought separately, and the cost is higher than deploying the application itself. Moreover, it is often encountered that there is not enough memory and the whole system crashes, which is particularly painful to troubleshoot.
Finding a better program
We've tried a variety of solutions.
I tried container services from various large cloud vendors (e.g., AWS, etc.) at first, but they requireDeploying a separate set of K8s clusters, maintenance costs are high, andConfiguration is complicated., we simply can't get it right and don't know what so many options mean.
Then tried Railway, an innovative PaaS platform from overseas, and they really did a great job with the user experience, but there were problems, even though it shielded the underlying details of theThere is no access portal to advanced features for advanced playersFor example, if I want to debug in a container, Railway doesn't currently support that. In addition, RailwayFor overseas use only, domestic access is slow to meet domestic business needs.
In the end, we chose to use Railway for our offshore environment, but we'll have to keep exploring for our domestic environment.
In the process of searching for a domestic deployment solution, we also tried to build our own K8s cluster. However, this process made us deeply realize the challenges of small and medium-sized teams in terms of operation and maintenance:
First, there is the cost of learning; K8s itself is a huge system with a lot of concepts and configuration items. Even for experienced developers, it takes a lot of time to fully master it. For a team like ours, which focuses on product development, this kind of investment is not very cost-effective.
Second is the difficulty of maintenance. Building a cluster is only the first step, and then you have to deal with various operation and maintenance issues:
- Node expansion and contraction
- Monitoring and Alarm Configuration
- Log collection and analysis
Finally, there is the issue of stability. Due to the lack of professional operation and maintenance experience, the cluster we deployed often had some unexpected problems. For example, a node was suddenly unavailable at a certain time, and it took a long time to troubleshoot before realizing that it was caused by insufficient disk space. These problems not only affected our development efficiency, but also negatively impacted the user experience.
As developers, we prefer to focus on the product itself rather than the maintenance of infrastructure. We needed a solution that would meet our technical needs and be easy to maintain.
Why Sealos?
After a number of attempts, we finally opted for theSealos Infrastructure program as a domestic environment.
Sealos' design philosophy is very similar to ours at Teable, in that it takes complex technology and makes it available to users through a simple interface, without limiting the space available to advanced users.
Specifically, Sealos addresses several key issues:
- Unified Management Middleware：comprehensive databaseThese are standard open source components, such as Redis and object storage, so you don't have to worry about being locked in. And these middleware can be deployed with a single click, eliminating the need to purchase and maintain a separate K8s cluster.
- Simple to use and easy to put away: Sealos masks the complexity of K8s and provides an intuitive interface for a simple and intuitive operating experience. At the same time, Sealos also retains the advanced features of K8s, so we can flexibly configure it according to our needs.
- Reasonable cost: Before, buying the cloud database and Redis separately was more expensive than deploying the application itself. Now I only have to pay for the application, whether it's the business application itself or the middleware, it's pay-as-you-go.
After getting on Sealos, the downtime for our own reasons hasn't happened since, and now theI've got something to throw at you.up, which is good haha.
In terms of input-output ratio.Sealos has made the ROI on my time invested in operations and maintenance incredibly high.I don't need to waste any more time on things that consume my life. Even if you have to learn a little bit of K8s, what you learn is directly usable, and what you learn is what you earn.
The implementation journey: from complexity to simplicity
Architecture Design Philosophy
When doing architecture design, we uphold the less is more philosophy. We try to use the simplest architecture to solve complex problems.
The overall architecture of Teable consists of the following components:
- Stateless applications
- Front and back ends are packaged in the same Docker image
- Supports horizontal expansion
- Easy to deploy and maintain
- core middleware
- PostgreSQL Database (Required)
- Redis Cache and Message Queue (optional)
- MinIO Object Storage (optional)
We purposely designed the degradation scheme so that the application can run even without Redis or object storage, only the horizontal scaling will be limited.
Standardized deployment process
Sealos App StoreIt's a great feature, and it's just like the macOS app store, where you can install apps with one click, except that the installs are distributed apps on the cloud.
With the Sealos App Store, we have achieved a highly standardized deployment process, using application templates provided by Sealos directly, deploying the application and all middleware dependencies with a single click, and automating configuration and connectivity.
Sealos also provides a kubeconfig file with specific permissions for each user, which we can use to connect to the cluster directly without having to configure it manually.
With kubeconfig, we automated the update of the application through GitHub Actions, also known as CD. The core workflow code looks like this:
The complete Workflow can be consulted:/teableio/teable/blob/develop/.github/workflows/
What used to take days to set up now takes minutes with Sealos, so we can focus more on product development.
Preview Environmental Automation
Now that you have kubeconfig, the preview environment is fully automated.
Similar to the production deployment, we create a preview environment based on the yaml template provided by the Sealos App Store, and apply it via kubectl to automatically allocate resources and configurations.
The yaml templates for all apps in the Sealos App Store can be found in theThis warehouse.Find it in there.
Now, here's our workflow:
- Development Submission PR
- Automatic creation of preview environments
- Tests and reviews are done in the preview environment.
- Automatically clean up resources after merging
We thought it must be complicated to set up the preview environment, but it only took a day or two.
You know what this means? Each development branch can have a separate test environment. After the test is automatically cleaned up, do not have to worry about. This kind of ability, even with a professional operation and maintenance team may not be able to get it.
Details are available:
- Create a workflow that previews the environment:/teableio/teable/blob/develop/.github/workflows/
- Preview workflow for environment cleanup:/teableio/teable/blob/develop/.github/workflows/
Looking ahead: Growing with Sealos
I'm honestly excited about the future of Sealos, and it would be great to see some of the features continue to be iterated upon.
Here are some of the things I expect from Sealos:
- Increased observability. We all know that the logging system is like a "black box" for the application. It would really help to have longer log retention time and more powerful search capabilities.
- Object Storage Optimization. The cost of object storage is still a bit high, and it would be nice to have some cheaper object storage solutions available.
- User Experience Enhancement. I know that Sealos has adopted a masterless architecture in order to reduce the number of faulty domains, all the available zones are autonomous and there is no unified console, each available zone is a separate domain name. However, there is still some optimization needed for this architecture solution in terms of user experience. The domain name of each availability zone is different, which is not easy to remember, and it would be good if it can automatically jump to the availability zone that users often visit.
In addition to that, I'm looking forward to Sealos going live in overseas environments so that we can completely harmonize our deployment solutions at home and abroad.
Choosing the right cloud platform and tools can help teams focus on product development while gaining enterprise-grade stability and reliability. That's the value of the cloud - democratizing access to technical skills and enabling every team to build high-quality applications.
We look forward to growing with Sealos and working together to provide better tools and services for developers.