Press "Enter" to skip to content

Developing an Effective, Flexible and Scalable Technology Stack

By the time I finish writing this blog, at least one of the libraries our project relies upon will have been updated. The world of web programming has taken huge strides over the past few years; if you were dormant or pursuing other fields, and have decided to return to the plain-old HTML/CSS/JavaScript combination – you’re going to have a bad time.

My first foray into the programming world began in summer 2012 when I opened a chunky book on the basics of PHP. There, I picked up a whole host of otherworldly concepts that I simply could not apply to reality. Nevertheless, as I became increasingly familiar with what was previously on-screen gibberish, I began fitting the pieces together. During this process, I came across a tremendous amount of “best-practices”, “do’s and don’ts” and “how to avoid being that programmer”. After a while, you pick up a bunch of habits, some great, and some meh. This tendency of “sticking with what you know” is what I like to call The Great Compromise of Being an Effective Programmer. If you went down the rabbit-hole of trying to be the best programmer in your field, you may never come out. You will certainly never complete a project (if development projects can ever really be completed).

Development Good Practice | XKCD
Good Practice by XKCD

Then, we throw development frameworks and bootstrapping libraries into the mix. Good luck convincing other developers that your stack is better than theirs. Therefore, what do we do when have to pick between major web-frameworks such as Angular.js and Ember.js, a whole plethora of database structures and hosting-platforms? Pick the one that takes you the least freaking time to learn, and just get on with it! Because from my experience, which is obviously completely un-biased and totally objective, you’re going to come across moments of complete despair, wondering how much greener the grass is on the other framework. Chances are, developers from those other frameworks are probably eyeing up your lawn just as much.

So this is the part where I tell you things aren’t so bad.

Moving on… here are some specifics on our stack and our reasoning behind each one.

Database management: Parse

Parse is a back-end-as-a-service (BaaS) and perhaps one of the largest in the relatively new industry. We toyed around with their API on smaller projects back in its early days, and whilst the technology impressed us, the pricing did not. Fortunately, before we started-up Synap, Parse was bought out by, would you believe, Facebook. Certainly a strange purchase by the social-media giant, given that Parse is solely a B2B for developers.

Backend as a Service Parse.com
Platforms supported by Parse

– Off-topic digression – An assumption by my Co-Founder, James Gupta, is that Facebook wants to keep an eye on up-and-coming companies, sweeping in to purchase them in their early-stages before they become too expensive *cough* Snapchat *cough*.

Why Facebook bought Parse.
Snapchat denies Facebook.

So post-sale, Parse is now incredibly generous in their free-tier allowances. We have no limit to our API calls per month. There’s no limit to how many classes (tables) or attributes (columns) we can have in our schema, so long as we don’t exceed 20GB of data. They have a file storage allowance built into the service, also capped at 20GB, more on this later. Finally, mobile push-notifications for up to 1 million unique users, with no limits on how often we send out pushes. It was definitely a worthy platform for a budget-strapped startup. Having said that, once you reach the upper-limits of the free-tier, the pricing soon catches up on you. The first paid-tier begins at $100/m and doesn’t exactly add much to your allowance. Put it this way, once we start paying $100/m, we’ll soon be paying $800-1k/m.

What we like about Parse

It takes a matter of hours to have your database scheme set up, the API and docs are ready for you – there’s literally no hassle. Comparing that to my old PHP/MySQL stack, let me tell you, I tore out far fewer of my hair with Parse.

Authentication, both for secure-password storage and social-integration is taken care of. They also include a number of predefined hooks such as email verification for new signups and a password reset email/link.

Parse has something called “Cloud Code”, giving us the ability to manipulate objects before and after they are saved, and write our own custom functions, all in JavaScript, hosted by them. This gives us an enormous amount of control over our data.

What we would like Parse to improve on

Regarding their Cloud Code, Parse is quite strict on how much execution time we are allowed. Now, I understand they have to keep their servers efficient and by enforcing us on a time-limit, we have to be extra careful with the quality of our code. But, it introduces a terrible uncertainty with tasks that are crucial for the inner-workings of our product. I think a better way to handle this would be to warn projects that are running over their execution time, rather than to cut off potentially irreversible tasks.

Parse also provide access-control over individual objects as well as whole classes. This is great, and would certainly belong in the previous segment of this blog, but for the following problem. Say we have a User class. Say we need to store sensitive information on that class, such as an Email and Telephone number. Now, if we make the object private, it becomes difficult to fetch the user. We have to create a custom Cloud Code function that fetches the user using a master key, then, delete all properties that are sensitive – returning the stripped down object. This is a common work around which should really be built into Parse. Ideally, I’d like to set all User objects are publicly accessible, and be able to set individual properties as “only visible with master key or for certain roles”.

Finally, I think their console logs are a massive pain in the .

Front-end Framework: Ember.js

There is a risk that I will write a thesis on my views about Ember. So I’ll focus only on how Ember works with our BaaS, Parse.

The Ember community is fantastic and incredibly fast-paced. Ember 2.0 came out a week ago (mid-august 2015), and we are about 2 to 3 major builds behind. This is where my early rant about not being able to keep up with the latest and greatest stacks comes in. Ember ships with its own REST API management library, ember-data, and to be honest, it’s pretty sophisticated. Whilst it allows you to customise the REST adapter, it can be quite a steep learning curve, especially if you’re new to programming, or new to model-view-controller (MVC) frameworks, or new to Ember, or a sane human being.

Ember.js was a better choice for us than Angular.js
Ember vs. Angular

For us, given that our REST API is structured by Parse, we needed an Ember-Parse library. Luckily, we found one, ember-parse-adapter. I’ve been working with this library for the best part of a year now and unfortunately, it has only been updated once during that whole time. As Parse updates its docs and as Ember updates ember-data, I am left with quite a mess. At this stage, I have had to extensively modify the library to match our needs. It’s not ideal, but it does the job. The down-side is however, we cannot upgrade to Ember 2.0 until the ember-parse-adapter is fully compatible with the new ember-data package.

What we like about Ember

Highly-opinionated and structured framework. Robust handling of routes and state transitions. Lots of plugins and 3rd party libraries.

What we would like Ember to improve on

Ember guide docs have been hit and miss, especially whilst new builds are released. It’s something they’ve promised to work on with this new major release. But we’ll have to wait and see.

The other major issue that is more general to ‘single-page web applications’ (SPA) than Ember alone, is search engine indexing. Put simply, when Google crawls our website, it gets a blank page because that’s really all there is. Ember, and other SPAs, load templates onto the DOM via JavaScript. Google’s crawl bots for various reasons, do not use JavaScript. The solution? Pay for services like Prerender. 

Server hosting: AWS Elastic Beanstalk, S3 and Redis

We picked AWS to handle our Node.js server. All it does is receives a call along the lines of ‘https://synap.ac/link/to/page’, ignores the ‘/link/to/page’ and simply sends our index.html which is stored on the Node server. Now, every time we update our project on Ember, we upload all our assets on an S3 bucket, links to those assets are referenced in our newly generated index.html and finally, the index.html is uploaded to Redis. Our Node server intermittently checks Redis for updates, and caches the latest index.html, spitting it back out to any url request. The actual page routing is done on the client, by Ember. Magic really. Currently, this setup costs us next to nothing and can handle an impressive amount of stress. Our server is not really having to figure much out, so it takes a lot of concurrent calls to cause it any trouble.

What we like about AWS Elastic Beanstalk, S3 and Redis

Cheap, flexible and scalable.

What we do not like about these three

This setup is not one that intuitive to inexperienced programmers. We had a lot of sleepless nights trying to get our head round this. If you’re in a similar situation, tweet to me @OmairVaiyani and I’ll talk you through it.

Other Services

Services we like to use as a development team:

Codeship: streamlines building, testing and deployment

Trello: light touch task management

Slack: light touch team communication

Intercom: user support and engagement, can get pricy but cheaper than hiring a support team.

XKCD A stitch in time saves 9
TL;DR by XKCD

Conclusion

Our technical stack is not the most sophisticated, rigorously efficient and unbreakably robust architecture that one could build. But, it’s better than what most tech startups can afford to build on their own. Our use of Software as a Service (SaaS) platforms has given us a massive leg up, and allowed us to focus on what we do best.

  • http://rb.sunglasses-hut.us.com/ ray ban sale

    Its such as you read my thoughts! You appear to understand so much about this, like you wrote the guide in it or something. I think that you just can do with a few p.c. to drive the message home a little bit, but other than that, that is excellent blog. A great read. I’ll certainly be back.

  • http://mobile.oksunglasshut.net/ oakley sunglasses outlet

    Saved as a favorite, І like your web site!

  • James Smith

    Considering recent Parse.com shut down how do I migrate my Mongo DB?

    • James Gupta

      Hi James, we actually found the migration process to be quite smooth! We completed it in about a week using the new Parse Server open source on Github, and hosting the database on Mlab (previously MongoLab) which is a hosted ‘MongoDB as a service’ platform, though you could set up your own if you wanted.

      We might produce a more detailed migratio guide in the future, but the official parse docs on this would be a great start!

    • Clark Thomas

      Hi James, I’m a developer and used to Parse.com. When I knew it would shut down, I began to look for alternatives and found http://www.back4app.com. I’m happy with my choice, there are several tools that serves me, and best of all, I managed to migrate my project in just 5 minutes!

      • https://synap.ac James Gupta

        Hi Clark – glad you also got through the migration process okay!

        Yes, we looked around at different alternatives and back4app was one of the ones that came up – it looks like a good platform and, I’d definitely suggest to people that they’re one of the ones worth checking out.

        We decided to go with a regular cloud hosting service (AWS / ElasticBeanstalk in our case) rather than jumping on to another Backend as a Service Parse alternative because, I think that model is shutting down generally.

        The BaaS market was a relatively new offering over recent years, mostly aimed at startups and individuals looking to start building apps and websites as quickly as possible, helping them to add functionality such as Facebook logins, manage NoSQL stores, security and push notifications etc in a really slick way, and abstracting them from a lot of the complexities that come with managing your own stack.

        Great idea, but Parse closed down, and a few years before that, the previous market leader, StackMob also closed down. And now, the non-BaaS hosts like AWS have caught up because it’s much easier to set up and manage secure, scalable architecture through them, using something like ElasticBeanstalk or Heroku. Similarly, for the other benefits a BaaS provides, there are now open source packages like Kinvey and, recently, Parse Server which you can deploy to your own stack… probably about as quickly as you could migrate them to another BaaS.

        Once you’re on your own stack, the costs are cheaper (unless you’re in Parse’s free tier, which was unsustainable anyway), you have more flexibiltiy and control over your infrastructure, and your data is non-opinionated and non-proprietary, so working with different formats, speaking to different services etc becomes much easier.

        I’m incredibly grateful to the team at Parse – I had a blast working on their platform for both serious projects and smaller side projects, and coming from more of a PHP/MySQL background it helped me get started on the ‘full stack javascript’ development environment quickly.

        But, to be honest, for the last 6 months Parse had become a concern for us – we had no choice but to wait for their development cycle to introduce features we badly needed (longer script execution times, automatic backups, webhooks, websockets, an EU/UK data centre), and it was getting to the point where we were looking to migrate anyway, but it would have been a major obstacle to re-create that infrastructure.

        Parse Server being open sourced was absolutely the best thing that could have happened, because it let us put Synap on our own infrastructure and start implementing these features!

        For companies in a different position, absolutely check out https://www.back4app.com/ – but, also consider a standard cloud host and running your own infrastructure on something like ElasticBeanstalk or Heroku because at some point, you’ll outgrow what a BaaS can offer you!

  • Rucha

    Hi, we have also migrated to back4app, but now unable to configure it in our app.Migration has been successful.Can anybody tell how can I get my app working again using Back4App..?