Tameem Iftikhar

DAG Factories — A better way to Airflow

2021-06-04T01:44:03+00:00

Using dynamically generated DAGs for scaling airflow

Every time you comment code — you’ve already failed.

2020-05-26T01:44:03+00:00

Hold your pitchforks and hear me out first…

Three simple yet profound ideas for rapid personal growth

2020-02-18T01:44:03+00:00

It just might be the inspiration you’re looking for

Slick Like Ruby, Fast Like C — Does Such a Language Exist?

2020-02-18T01:44:03+00:00

Yes, yes, it does

The Ultimate Guide to Blazing-Fast Performance in Rails

2019-12-29T01:44:03+00:00

Ruby on Rails is a tremendous framework when you want development speed for your project or startup. It’s useful right out of the box and comes with a plethora of behind-the-scenes magic to make your life easier. However, it’s not considered the fastest framework out there in terms of performance. You will find examples of individuals and companies drifting away from Rails in favor of something else. Despite this, there are many companies out there who have succeeded in scaling Rails and found success — just take a look at Airbnb, Github, Gitlab & Shopify.

So before you jump ship, you should consider keeping performance at the forefront of your mind when working with Rails, and you can succeed too. This article aims to list the most important tips & tricks I’ve learned over the years to make Rails run at blazing fast speeds and scale to millions of requests per minute.

General Tips

First off, there are some general tips to implement in your Rails project to set yourself up for success.

Set Up an APM

You can’t improve performance if you don’t measure it first, making it necessary to have the right metrics tracked and monitored. You should be tracking load times, request times, and database query timings amongst other things. Personally, I’ve found New Relic to be one of the best APM tools for Rails but it is a tad bit on the pricey side. A more affordable alternative is SkyLight’s free trial.

Stay Current

I know personally how horrifying updating the Rails version for your project can be if you left it stale for too long, and I sympathize with anyone who had to go through this ordeal. So do yourself a favor and try to keep in sync with the newer versions of Ruby and Rails. This will help you skip the pain of jumping multiple versions and ensures that you have all the newer performance enhancements.

Pareto Principle (the 80/20 rule)

This is a well-known rule when it comes to developing software — The Pareto Principle. It’s also known as the ‘law of vital few’ and states that for most events, 80% of the effects are generated from 20% of causes. The idea behind this is to not waste time on micro-optimizations while you have bigger issues that need resolution. You won’t accomplish much by shaving off milliseconds in serialization if your database queries are extremely slow. So pick your battles carefully.

Keep it Lean

Rails is backed by an amazing community behind it and a library of gems that can easily help you accomplish complicated tasks. But it’s easy to get carried away adding gems to your project, causing bloat. Be careful while selecting what gems to add to your project and try keeping your dependencies lean.

Be Lazy — Do it in a Background Job

Whenever you need to do anything complicated or long-running, consider throwing it into a background worker — sending an email, push notifications, uploading pictures and the like. Minimizing work in the main thread will ensure a snappy response for users. The good thing for us is that Rails has multiple options to achieve this easily — Sidekiq, Rescue or ActiveJob.

Change Your Serializer

If you have an API in your project you are most likely using the ActiveModelSerializers gem for serializing your data. I would strongly suggest switching over to the fast_jsonapi by Netflix. This gem is a whopping 25 times faster than the default ActiveModelSerializers and I can personally vouch for that from experience.

Cache Me Outside

Sometimes if you have a lot of static data or you can’t make things faster, another alternative is to use a cache. Rails makes this incredibly easy to do right out of the box. Here is an example of how easy it is to do a cache with an expiry time:

  # Lets say you have some categories you offer in your project.

  Rails.cache.fetch("categories", expires_in: 5.minutes) do 
      Categories.all.map(&:name)
  end

Active Record (Database)

ActiveRecord is the magical ORM (one of the best ever to exist) offered by Rails. It’s easy to get caught up in the ease of use without understanding the details which can lead you to bottlenecks in the future.

Use Thy Database

This is something that goes unnoticed or unimplemented by a lot of developers as they rush to do everything in code or maybe they are intimidated by writing raw SQL. But whenever possible or convenient, you should use thy database. Processing and sorting data structures in Ruby can chew up CPU time, while your database can do this without breaking a sweat.

Peek Behind the Curtain

Don’t just get enamored by the magic of active record without worrying about what’s happening behind the scenes. In order for you to sweep out performance bottlenecks, you need to look at the actual queries that get triggered and understand them. In the development environment, Rails will print out all the queries getting run which allows you to notice any unnecessary queries. Also, here are a few tricks that will help you dig in a little deeper:

SQL Wizardry

Don’t worry, I am not about to suggest you write everything in raw SQL queries. But learning the basics of SQL and database design will help you better understand what’s happening under the covers and allow you to optimize the queries as needed. It’s a valuable skill to have as a software developer and will go a long way in your career.

Be Stingy

One way to make your queries more efficient is to only select what you really need. Instead of doing a SELECT * specify which columns you need the database to retrieve. By default ActiveRecord selects everything but you can leverage select or pluck to fix this problem.

N+1 Queries

This is a classic problem. If you are loading a Blog from the database and then try to find all the comments for that blog by looping through the records, you are forcing Rails to run a query for each of the comments. This can be eliminated by preloading the comments Blog.includes(:comments) and helps avoid the N+1 query problem. Pro Tip: take a look at Bullet gem to help you find any N+1 query problems.

Combining Queries

When working with bigger teams with multiple developers on a project, sometimes you will notice every person touching the codebase might be adding queries throughout the code path. More often than not, these queries can be combined into fewer queries at the top of the code path. This ensures there are no duplicated queries and allows the database to strategically do the heavy lifting.

Index, Index, Index

This is a database best practice that shouldn’t be ignored. If you don’t index properly you will negatively affect database performance and cause unnecessary table scans. When building a new feature think ahead to what will be queried in the project and attempt to add the correct indexes. If you have an existing project you can always use Active Record Doctor or LolDBA to sniff out missing indexes.

Migrations

When you are running at scale, and you have millions of records in your tables the normal way of running migrations in Rails will, unfortunately, fail you. They will either error out or lock tables for extended periods of time, taking your site down. Having dealt with this in the past there are two tools that will solve this pain point for you:

Gh-ost: This is a triggerless online schema migration solution for MySQL and the only tool that worked for me at scale.
LHM: This will migrate your tables while they are still online without locking your table.

Extreme Tip: Use Raw SQL if you dare

If there is an endpoint where you desperately need performance, you can consider combining all the Active Record queries throughout your functionality to a single massive SQL query at the top to pull out any records you need to fulfill the use case. Disclaimer: Yes, it’s hard to maintain a massive raw SQL query, and it’s also less readable. But I did mention this is only in case of desperation.

Servers & Hardware

When deploying a Rails project there are a few things to remember regarding the underlying infrastructure and architecture.

Choose a Scalable Architecture

With cloud infrastructure having come such a long way, gone are the days of having to build bare metal servers in order to scale your application. When deciding on your underlying architecture for a Rails project you should utilize a scalable cloud-based system. As an example, you can use AWS Fargate or Kubernetes to automatically scale your dockerized Rails application as needed.

Brotli Compression

Brotli is another compression algorithm based on gzip with multiple improvements and a better compression ratio. It is now supported with most web servers and adding it is an easy way to optimize the compression speed. And well, who doesn’t want free web performance improvement. (Reference: Brotli vs Gzip)

Give it More Juice

Rails is notorious for using a lot of memory, especially if you have multiple puma workers running. So don’t let your application go thirsty, and give it some juice from the start. You can utilize memory-optimized instances on your cloud provider to set you up for success. And don’t forget to keep an eye on the swap usage on the server.

Tuning

Since Rails 5 the webserver for Rails has been switched to Puma, and by default, it will only run one worker. As soon as you set up your Rails deployment make sure to increase the number of workers to the available cores on your machine or something more reasonable.

HTTP/2

If you are utilizing a reverse proxy like Nginx make sure you have the HTTP/2 option turned on to give you all the performance advantages you get from it.

The Cherry on Top (Miscellaneous)

We are almost at the finish line — this section is all about some extra tips for performance.

Don’t Use Dynamic Methods

Rails magic comes with a price. A few of these methods consume a lot of resources and it would be better to skip using them. As an example, the find_by() and find_all_by() are kind of slow because they need to run through method_missing and parse the name against the list of columns in the DB.

Throttling

As you scale you are bound to run into malicious attempts on various endpoints that can’t be cached and will cause expensive operations to chew up resources. To get around this, I would recommend adding a gem like rack-attack to implement throttling on endpoints like login, reset password or sign up.

Know your O(n) vs O(1)

As the data set size increases we need to heed the impact of our code in terms of time complexity. In a scenario where lots of data needs to be processed or looped over, consider using a hash instead of defaulting to arrays.

Always use a CDN

All your static assets should always be fronted by a CDN and make sure the cache policies are reasonable. If you want granular control over the cache invalidation, you can look into using e-tags and Cache-Control.

Advanced

In extreme cases, you can consider some advanced optimizations. I would argue if you’ve reached this point it might be time to consider other languages for certain parts of your project that are computationally intensive, so I will keep this section brief.

Brush up on C

The main implementation of Ruby is written in C, which allows you to rewrite the slow part of your code in C as an alternative — e.g. if you are doing crypto algorithms for generating certificates. If you aren’t so excited about the idea of dwelling into C code, you can leverage third-party gems written in C bythe Rails community.

Faster Background Jobs

With an increasing background workload, you can switch your background workers to a more performant language. By utilizing queues like Redis or Amazon SQS, the workers can be decoupled into their own microservice. Checkout this Sidekiq compatible go-workers library or a version of sidekick written with Crystal as two options.

Try Faster Versions of Ruby

There are a few implementations of Ruby aiming to increase the performance. If you are interested in these variations, take a quick look at Truffle-Ruby or JRuby.

The Bottom Line

If we look at companies using Rails we see that they were able to utilize the amazing development speed of Rails to put their customers first, and also managed to improve performance as they scaled. In this article, we talked about multiple tips & tricks for increasing performance. I hope you found this guide helpful. Till next time.

Love talking tech or startups?

You’re in luck, me too! If you want to chat about cutting-edge technology, entrepreneurship or the perils of being a startup founder, find me on Twitter or on LinkedIn.

Bougie to startup founder — Tips for living on a budget.

2019-12-16T01:44:03+00:00

So you’ve finally decided to quit your cushy job to pursue your own startup dreams. Good for you champ! I know personally how daunting it can be and requires some habit and lifestyle changes. You might be living off your own means while you wait to raise a round of funding and build your first product. Now it’s time to cut some Uber bills, delete a few delivery apps and break up with toxic subscription providers — the things we do for our dreams . . . I’ve personally been through this process and have compiled some tips to get you on your way to becoming a personal finance wizard(ess)!

Budgets, Budgets, Budgets

Some of us have been through this before — downloading various budgeting apps with a newfound enthusiasm for personal finance, yet we eventually never follow through. I agree that budgeting is time-consuming and boring but unfortunately, it’s in your best interest to stick with it. Here are a few tips to make the transition easier:

Tracking budgets manually is difficult so automate this process by finding an app that works for you. (I personally use Mint)
Start by creating bigger buckets of budget categories — rent, bills, food & others. Take a few iterations to hone in on the right amount of the budget for each category.
Once you’ve got the hang of those budgets, you can start to create more granular budgets in terms of drinks, groceries, clothing e.t.c.

The most important thing to remember while you take up budgeting is to think of it as an iterative process. Just like any habit, it will take time to establish it. You might set budgets and not meet them — but the key is to keep readjusting both the budgets and your spending habits until you find the sweet spot.

Detective Mode Activate

When you are getting into this new habit, it is beneficial to sit down and sift through your bank statements with a detective’s eye. While doing this exercise the idea is to:

Get intimately familiar with your monthly spending and habits.
Look for any unnoticed or unnecessary subscriptions or charges.
Banking fees that you might be paying for your account (there are options to get accounts that have no monthly fees).
To come to terms with your bad spending habits and accept them as they stand now.

Reduce your monthly bills

We keep paying for internet and mobile providers every year without wondering if there is a better deal out there. If you’ve been with your provider for a few years, it’s time to get some loyalty discounts or possibly find another provider. Seriously, it works. I saved a good chunk of change by calling both my internet and mobile providers and asking them for better plans.

Subscription Intervention

I know how much you love your subscriptions but let’s be honest- do you really need all of them? Maybe it’s time to break up. Start by going through your list of subscriptions and figure out what doesn’t spark joy. Some good strategies for better handling your subscriptions is to:

Keep only one subscription service in each category that is important to you — music, video, fitness, e.t.c.
Set a limit for the amount of money you want to spend on subscriptions in total and cut them until you reach the goal.
Cut all of them, and slowly bring them back online if you can’t live without it. This one is a more aggressive strategy.

Don’t “Skip the Dishes”

If you haven’t already, another lifestyle change that can save you money while also contributing to healthy habits is to start meal prepping. Keep in mind that when beginning this habit it can take some learning and getting used to. So you can try two different strategies.

Use a meal service

If you don’t have the cooking skills or the time to spend on meal prep a good option might be to consider ready to cook meal boxes like Chefs Plate, GoodFood, or any of the others on the market. They are a good way to level up your cooking skills while enjoying delicious meals at home.

Learn to meal prep

The other strategy is to get your hands dirty and learn how to meal prep properly. To get you started off on the right foot:

Decide on meals and recipes that align with your health goals and are easy to prepare and store.
Pick ingredients that can be used in multiple recipes that have a longer shelf life.
Pick a day for preparing the majority of the meals.
Bring variety to simple foods like baked chicken by having a good repertoire of spices.

Pay off credit card

Interest on most credit cards is through the roof so before you quit your job, make sure you pay off all your credit card debt. Ideally, you should never have debt on your credit card, but life happens and sometimes you have no choice. In the future, try and stick to buying things you can pay for in cash at the moment instead of depending on your credit card.

The Bottom Line

It won’t always be easy, but you’ve already taken the first step by looking into articles like this one. Stay vigilant and dedicated and it will become more natural with time. And yes, you can reward yourself once in a while with a night out, just don’t go wild. You’re not a millionaire. . . yet.

Love talking tech or startups?

You’re in luck, me too! If you want to chat about cutting-edge technology, entrepreneurship or the perils of being a startup founder, find me on Twitter or on LinkedIn.

Authentication with React Context

2019-11-11T01:44:03+00:00

Most react apps require the concept of authentication and storing user information. In these scenarios, you might end up with various components in the app that all need access to the same user information like the avatar or display name. On top of this, a few of our components might want to update this information and make it be instantly available everywhere else.

Some go-to solutions for this use case could be:

Add a package like redux to your project and add bloat for a simple use case.
Pass the information using props all the way down the tree, and create “props hell” for yourself.
Use the observer pattern by using something like PubSub JS.

As of new React versions, there is a better way to accomplish this by using React Context (https://reactjs.org/docs/context.html). In this post, we will run through an example of how React Context can be used for this specific case. So let’s get right into it.

Consider the following sample piece of code. We have a dashboard app that has two components somewhere in our tree: NavBar and UserSettings. Both these components need the user object and we can pass it through the props. This snippet is the extremely basic use case, but imagine having components nested deep in the tree and the pain of passing the prop all the way down.

This example and more complicated versions of it can be handled well by utilizing react context to store the user information, and all the components within the app can then use it on a need-to-use basis. It’s time to run through a step by step example.

Setting up a User Context component

First of all, we will setup the react context. To keep our code clean, we will define a provider and consumer in the same component file. Create a new folder for context and create a new file /src/context/UserContext.jsx.

Code Break Down 🕺

In the code above we define a new user context by using the createContext helper and specify that it will contain a user object and a updateUser method that will be used to update the user data.

We then define a UserProvider class that wraps any children in the context provider and keeps the required object in the state.

    // This provider will wrap the rest of the tree and we pass in the // user in the state and the updateUser function as well. 
    <UserContext.Provider
      value={this.state}
    >
       {children}
    </UserContext.Provider>

At the end of the file, we also export the Consumer in the same file.

    export const UserConsumer = UserContext.Consumer;

Now that we have a UserProvider component we can run through a practical example of how it can be used in our application. To add a little complexity to our pretend example let’s assume the TopNavBar and UserSettings components are deeper in the tree and not in the App component.

Setting up the initial value of a user in the context

We need to set up the initial value of the logged-in user in our main App.jsx component. This way as soon as a user logs in, the context is filled with the required user data.

Code Break Down 🕺

In the above code, our app component makes sure it pulls the logged in user’s information from the server and stores it in the state. Then the rest of our code gets wrapped in the UserProvider that we created earlier. Note that we are passing the UserProvider with the user prop to set up the value of the user in the context.

    <UserProvider user={user}>

Use the context in any component

Now that our app component wraps everything in the UserProvider component and setups up the initial value of our user, we can go ahead and use this in our components. Let’s circle back to the TopNavBar component and implement this.

Code Break Down 🕺

Alright, we are almost at the finish line. In this component, you can see that we are loading the UserConsumer from our component we created earlier.

    import { UserConsumer } from ‘../contexts/UserContext’;

IMPORTANT NOTE: In order to use the context in our component we need to wrap our component in the UserConsumer. I like to pass in the context as a prop to be used in the component. That’s what this next code snippet right here does. We wrap the component in the UserConsumer and pass the state to our TopNavBar component.

    const withContext = () => (
       <UserConsumer> 
         {state => <TopNavBar context={state} />} 
       </UserConsumer>
    );

    export default withContext;

Within the TopNavBar component, we have direct access to all the user info. We read it in our constructor and set it in the state to be read again.

    // Get the user from our context
    const { context } = this.props;
    const { user } = context;

    this.state = { user };

Updating new user info in the context

Now you might ask, what about updating the user information when the user uploads a new picture? Don’t worry about it, I got you. Taking the TopNavBar component again we can access the function passed into the context. Just like we got the user info we can access the update function from context and call it to update the info.

    // Get the user from our context 
    const { context } = this.props; 
    const { updateuser } = context;

    updateUser(newUserInfo)

Summary

In this post, we took a look at how React Context can be used to store global information that needs to be accessible by multiple components. It helps us write cleaner code without props flowing in all possible directions.

I hope this was helpful, and if you have any comments feel free to reach out.

Introduction

2019-08-10T01:44:03+00:00

Hi everyone! I finally did it, I went ahead and took the time to setup a website and a blog. In true techincal nerd fashion, I had to set it up in Jekyll and deploy it with an automated pipeline that has deploy previews and split testing. Overkill? Maybe so. But hey, that’s the benefit of being a techie after all.

So what am I gonna write about? Well, that part is still undecided and will come to me as the ramblings show up in my head. In the upcoming months you can hope to find tutorials, lessons I learned as a yong entrepreneur and other thoughts.