Scalability should be an important factor when choosing a platform or building your community. Different community types face different challenges in terms of scaling and hosting their communities. From compliance issues with GDPR to load balancing for new product launches, your community’s hosting should be carefully thought out. Consider reliability, availability, and geographic location when choosing your hosting provider. Larger communities may wish to progress from virtual hosting to dedicated hosting. Take a video game community, for example, they may have a 10 of thousands of users, and up to a thousand concurrent users at any given time, but when they launch a major update or expansion they’ll need to have the flexibility to accommodate the large influx of traffic. Before your community members get their pitchforks out when your community is down, consider the following for scalability.
Server configuration and redundancy
A single codebase will be the easiest way to start. If the webserver, database, and background processing are all hosted on separate servers - a single code base approach will be simple and should scale for most situations. Use a purpose-built storage platform for files such as images, and videos to provide high durability and availability. Files should not be stored on the webservers’ disk in case it needs to be replaced and to protect against issues related to a full hard disk. A Content Delivery Network (CDN) can be used to deliver static content to users who are not near your data center.
As your project scales, the single code base can be broken up into multiple services. Breaking up your code base needs to be planned ahead, and shouldn’t be done at the least minute, otherwise, you risk cost overruns and downtime in your community.
Server configuration and deployment should be automated. This allows for servers to be quickly replaced in case of a technical issue and quickly resized to facilitate scaling or cost management. Automated deployment can also make it straightforward to maintain a staging server (for live testing of new code) and to clone production for quality assurance purposes.
Database system and size
Follow database best practices such as indexing, null constraints, foreign key constraints, and others. Databases are influenced by the nature of the data they handle and the actions taken with the data so after establishing your database, be sure to monitor it closely to log slow queries and growth patterns. If your system will collect personal identifiers such as name, email address, birthdate or phone number remember to evaluate where your data will be stored, and who your data will be collected from to ensure your database will be compliant with applicable privacy laws. Ensure application data queries are written carefully so as not to introduce n+1 issues. Preload data whenever possible for optimal performance. Lastly, be sure that as your community grows you monitor your database. Log information should show slow queries and be able to help you to understand data growth patterns.
Backup operations and storage
Data backup should be automatic, frequent and verified. Restoring backups should also be automatic when needed, and you should test the process of restoring backups periodically to ensure that it will work when you need it. Storage should be somewhere with high durability, availability and appropriately controlled access.
Where data is stored is also an important consideration to ensure legal compliance for your community. The location of data storage must be disclosed to participants. In particular, some countries will require you to have specifically safeguard personal data stored by the system and in some cases, this will not permit storage in the United States. If you collect personal data on European nationals, also ensure you are familiar with the requirements of the Global Data Protection Regulation (GDPR).
Most hosting providers offer tiered costs which are influenced by a lot of considerations with respect to how your system is configured. Typically this involves different fees based on your server size, your storage systems, and requirements, whether you are using virtual or dedicated machines, and the network bandwidth you use for content delivery. These costs are dynamic, so it’s important to consider that they may vary from month to month and also as your community grows they will scale with that growth.
There are two considerations here with respect to scaling. First, as you scale and acquire more customers you will need more staff who can integrate with them and address support needs. Ideally, at some point, you will need to hire dedicated staff who address support needs using support ticket tooling. It’s not enough to have support staff only, you’ll also need to remember that your development team also needs to grow so that fixes will match support needs. It’s a good idea for your support ticket tooling to be able to integrate into the development process so that fixes can be prioritized and addressed in order of need. Lastly, as you start to gain valuable insight into customer needs and perceptions through support interactions you may want to consider hiring a Product Manager who can turn this data into useful product roadmap updates to ensure scaling is sustainable and that your platform stays fresh and current to the desires of your user base.
Choosing a SaaS Platform
Choosing a SaaS platform gives you the advantage of having everything managed by one vendor. However, like this document details, when data protection considerations and sensitive customer data is gathered, you will need to trust your vendor to properly follow guidelines and laws. The other consideration when looking at a SaaS platform is the additional markup you end up paying for having a third party host your community. Community platforms do not own their own data center, in essence, you're just paying for Amazon or Azure hosting. If you’re creating a smaller community, and don’t have access to dev resources, then, of course, it makes sense to look for a hosted option, otherwise, it typically ends up being less expensive to take care of the hosting and site management yourself.
Hosting and scaling a community should be no more different than building and scaling a complex website, or your web app. So long as proper planning for traffic and regulations is done ahead of the community launch, you shouldn’t run into trouble.
Ever since Slack and Dropbox took off with their low acquisition cost model of virality, Product Owners have been looking for ways to create a strong network effect in...
Cloudinary is a SaaS product that provides, true to the name, a cloud-based digital asset management system. It is intended for any size web platform that wants their...
Integrating Stripe with any framework can be done with relative ease, and Stripe provides great documentation for integration and testing. In this post, I will discuss...