This blog is about how my thought process changed in the 3 months of starting my job. It's an unstructured outpour of my thoughts.

Build for Enterprises

I've built projects aimed at general users and developers. My approach to any problem would initiate a thought process with "How can this benefit/work for me?" in an individual capacity, for example, billing, I'd prefer project-based billing. My thought process would be "I'll pay for the specific projects I want and that's it". Supabase switched to org-based billing this year and I was annoyed at it because again, I'll pay for the projects I want. Underdog had org-based billing when I joined.

There was a bigger picture to this that I had yet to see. Org-based billing is preferred by enterprise and institutional users where there's more than one developer. It helps them have a single abstraction on their usage and billing I assume there are more things as well. The service provider can also structure their offered services to allow n number of projects under a plan for an org, team usage, and much more.

Rather than catering to individual devs, build for enterprises. Think of what you'd prefer if you were at an enterprise level looking to use a service. If you cannot/fall short, talk to someone who has already built, is building, or at an enterprise user.

Learn Testing

I cannot stress how important testing is. Not giving it enough emphasis during my college days is a regret. At Underdog, every function, service, and module has a test. The code is thoroughly unit-tested, reviewed, and then deployed. When I started working on the codebase initially, it was challenging for me to navigate the codebase and find and work on things. I opened the code and rammed my head straight through trying to understand it and work on it, expecting comments/docs to understand what the code does. I didn't bother looking at the tests.

But hey, there are unit tests to test what the code does. So, I could've just looked at the tests to figure out what the code did. I started by looking at what gets triggered when an API call happens to our services and working my way downward to find its core logic and work on it.

I managed to implement the first functionality assigned to me this way and made a PR. I was happy and confident my code worked and it'll get merged easily. The first question asked during my PR review was "Where's the test?". I had to write a test for the functionality I had added.

I had never written tests for my projects in the past, guilty as charged, it had bit me back. I opened the tests, understood them, worked my way through, and wrote tests for my work.

Had I focused on tests earlier, it'd have been easier for me to work and my colleagues to review. If you're reading this, as painful as it might be, write those damm tests.

Don't Reinvent UX, Study Others

While working on Passport, which is a service to facilitate wallet-less NFTs, we were working on adding email UI/UX to it. I had the liberty to design the UI, so as a dev, I did what was easy and fast at the beginning. As things progressed, I realized that the intended users for this service are primarily non-devs, with a mental model of email services.

The UX I had in place did the job, however, it didn't quite fit the mental model of an email UX the users will have in mind. I took some feedback from the non-dev friends to confirm this. Now I had to re-engineer the UX. This time, I opened popular email services studied their UX, and implemented a UX that'll maintain the mental model users have when they hear email.

Understand the Problem at Hand

Understand the problem at hand before jumping into solutions. I'm guilty of this, and it's a habit of mine where I partially understand a problem, jump into writing code, take a step back when a roadblock is hit, and understand the actual problem at then solve it.

While understanding the problem, I simultaneously start thinking about possible solutions and get the excitement to start implementing, however, lately, this dopamine rush has been trouble causing.

A simple strategy I've devised to tackle this is to:
- write the problem
- try to explain it to a 5-year-old

This has helped me calm my nerves and focus better.

Don't Overengineer

In the past, for my projects, I've designed to handle a scale of millions but failed to get even 50 users for my app. This repetitive pattern, when implemented in a complex codebase, causes complex overengineering with multiple levels of abstraction.

I'm not saying building for scale is bad, I love thinking about how systems and code are built & maintained at a big scale. But complex doesn't mean scalable and maintainable.

The solution I found to this was:
- Learning about clean code principles
- Discussing my thoughts and solutions with peers and getting feedback

Don't Overdocument, Rewrite

This AVL Tree code I wrote in 2020, has comment explanations for everything. I religiously comment and document all of my code till date and expect a similar level of documentation. It frustrates me when code doesn't have comments for everything to spoonfed me. This has also been a barrier for me to work on open-source codebases.

Why is the code overdocumented? Because it's not readable.
Why is it not readable? Because I'm not aware of the practices toward it.
What's the solution to this? Learn how to write clean code

So now I'm reading a book on how to write clean code. The book hit me like a truck. I make all of the rookie mistakes mentioned in it. Comments are great wherever they're helpful, but they're not compensation for bad code.

I now try to write code that follows a good set of clean code practices and refactor until I'm satisfied with the code or frustrated enough to not give a damm about it.

Conclusion

The year has been full of learning. These are the most recent and I feel will be impactful. I'm grateful for the team at Underdog Protocol for my immense growth in the past 3 months.

This blog aims to be a checkpoint as I learn, upskill, and progress ahead. When will the next version be out you wonder? When I have enough points to span in a blog.

Hit me up on Twitter/X to talk about anything tech.

MSOL also gives you an NFT for the SNS you create as proof that you own it.

An SNS can be up to 20 characters long and can have letters, numbers, and hyphens. For eg. foo-bar

MSOL uses Vercel KV Redis database to store the SNS making it fast with average SNS lookup times of ~250ms.

MSOL provides Web & API lookup for your SNS for easy sharing & development

Demo

Public Key Lookup

There are 2 ways to lookup your key:

1. The web portal: In your browser, visiting <base-url>/<sns> will show you on the website whether the SNS exists or not.

2. Via API: To get the API key via an API call, hit the <base-url>/api/sns?sns=<your-sns>. For eg.

curl --location 'http://localhost:5173/api/sns?sns=foo-bar'

Sample Response

1. SNS Found

{
    "status": "MSOL_SNS_FOUND",
    "data": [
        {
            "publicKey": "<public-key>",
            "sns": "foo-bar"
        }
    ],
    "error": null
}

1. SNS Not Found

{
    "status": "MSOL_SNS_NOT_FOUND",
    "data": null,
    "error": null
}

Supporting MSOL

MSOL aims to take away the hassle of remembering your Solana Public Key.

MSOL aims to always remain free for its users.

Due to a lack of financial resources, MSOL currently supports only Solana Devnet.

Please get in touch if you want to support MSOL.

Costs to be covered for now are:

• Vercel for hosting & KV DB

• Underdog for NFT

• Images for NFTs

• Domain

End Note

Check out MSOL and create your SNS and tweet about it.

MSOL is created by Wilfred Almeida, get in touch.

The internet connection we had was a non-business connection under the founder Anirudh's account. We decided to upgrade it to the Airtel Business Broadband Plan. We decided to proceed with the INR 799/m plan which satisfies our usage. Upon doing the formalities and Airtel's personnel visiting the office, we received an email about the service in which it was mentioned that we'll get a free static IP for our connection.

Most of the company services are in the Minimum Viable Product (MVP) stage as of this writing, where spending money on cloud deployment solutions didn't make much sense. Since Airtel's email promised us a free static IP, we decided to set up an in-house server for our deployments, Linux based development environment, and centralized file hosting. We purchased the hardware and the connection was upgraded.

Airtel's Misleading Email

When we tried to redeem the free static IP from the Airtel dashboard, it bummed us that for our plan the free IP is not available. To get a static IP we had two options from Airtel:

1. Upgrade the broadband plan

2. Pay them an additional INR 99/m for the static IP

Our communication attempts with Airtel were in vain with no solid accountability for the email from their side.

We had planned the architecture of our systems, invested in the hardware, and most importantly, spending money on the cloud for trial and testing didn't make sense.

Enter DDNS

Upon exploring our options, we came across Dynamic Domain Name System (DDNS).

Dynamic DNS (DDNS) is a method of automatically updating a name server in the Domain Name System (DNS), often in real time, with the active DDNS configuration of its configured hostnames, addresses or other information.

~ Wikipedia

Since it seemed to eliminate our need for a static IP, we decided to go ahead with it and not purchase a static IP.

Our Network Architecture

Since Airtel's router/access point didn't offer many features and control, we decided to use a Ubiquiti Dream Machine (UDM) lying around in our hardware treasury as our primary network controller. We disabled the DHCP of the Airtel device and turn off its wifi and use it only as a medium to interact with the fiber optic network cable.

To our surprise, Airtel has locked down the firmware so much that we couldn't disable the wifi from the dashboard ourselves.

We gave a WAN static IP to the UDM and made it our primary network controller.

Airtel's unsolicited control

We asked Airtel about turning off the wifi and Airtel turned off the wifi signal of the physical device IN OUR OFFICE from THEIR SIDE via their control panel. This immediately made us question the access and control Airtel has over us and raised privacy concerns. But without much choice, we kept it aside and moved on.

Airtel and DDNS Providers

From our research, no-ip seemed the most lucrative provider for free and reliable DDNS. But to our surprise, Airtel doesn't support no-ip as a provider in its dashboard. The available providers were either:

1. Paid and expensive

2. Their sites were 404

Stunned by this behavior, we reached out to their support who gave us vague responses like "your query has been forwarded to the respective team" for 2 days. The same individuals who turned off our wifi swiftly needed a team to respond to no-ip as a DDNS provider.

Personally, I find this hard to digest🤔

Finally DDNS

With the UDM supporting no-ip as a DDNS provider, we signed up on no-ip and tried to set up our DDNS. With some initial issues with IP updates and other things, we managed to get DDNS working.

When we hit our DDNS link, the Airtel router's management dashboard page is typically visible at 192.168.1.1 loaded. Surprisingly Airtel had port 80 open for us or didn't block the traffic yet or whatever, we had some progress.

Port Forwarding

We expected traffic to flow like this:

hit to our ddns URL -> port of Airtel -> Port of UDM -> Port of Server

But this isn't how it was happening. After quite some time spent playing around with different port forwarding and firewall config, we discovered that there was no port forward needed in the Airtel device. Adding firewall rules in the UDM did the trick and finally traffic hit our server 🥳

Accessing from Our WiFi

We hosted a sample application on a port to test out the DDNS working. Trying to access them while connected to the in-office wifi didn't work. Accessing from a different internet connection somehow worked.

Then we discovered that usually with DDNS this happens and the solution to this is more work.

Checkout this SuperUser thread to learn more about this.

The solution to this as far we understood was to make a DNS entry in our local DNS to resolve the DDNS URL. We planned on setting up Pi-hole for ad-blocking and local DNS.

DDNS Performance

For a base API of hello world, without any DNS or any other caching, it took less than 100ms in response time. So the DDNS setup for us was fast enough.

Enter NGINX

NGINX is a web server that can also be used as a reverse proxy. We set up NGINX on our server. As mentioned earlier, on our port 80, Airtel's login page was loading. When we tried to access it after some days when our NGINX was set up, we were not surprised to see that it didn't load. A possible reason for that we believe is Airtel blocking the port 80.

We changed NGINX to run on a different port and we were able to access it via OurUrl:port

We could set up a port 80 redirect in no-ip wherein it would redirect the traffic for port 80 to some other port. But, that was not free and we needed to upgrade to premium for that. And the base premium rate as of 30th June 2023 was $1.99/m.

SSL & Domain Issues

We needed SSL. SSL from no-ip would cost us. We also needed a subdomain for our server under our main domain. Doing these things was complicated, tiring, and time-consuming.

The Realization

We went with DDNS to save the INR 99/m, but we were now at a point where it was costing us billed time and effort to get it all up. We ultimately gave up and got a static IP from Airtel. Following is our reasoning:

1. Is it worth it?

   As a business, saving every penny you can is important. But if this little saving is costing in terms of time and effort, then is it worth it?

2. Figuring it all out

   No one in-house had the practical experience of dealing with DDNS and the things discussed here. It was a figuring-out journey where we had to sit and look things up and try it and get it working. This costs time.

3. People working on it

   With the size of the organization, we had only one big brain (me ofc) working on it all. Only that one person knew what all things were done to get it all working. So the documentation, knowledge transfer, and debugging would all depend on the person, whose talent is better utilized in other important aspects.

4. Delays Introduced

   For in-house machine learning workloads, we planned to put it on a VM on the server hardware with GPU for efficient and multiuser workloads. Also, to allow access to company and in-office resources for the hybrid workforce, we needed a firewall and VPN config. All of this depended on our server which got delayed.

   Apart from this, ready-to-be-deployed applications got delayed which needed to be showcased to investors and clients.

5. Why not Cloud VM?

   Cloud VMs cost money. Given that we don't have production load and revenue yet, it doesn't make sense to invest money into those. Also, we have machine learning workloads that need GPUs. Cloud GPUs are quite expensive. We'd rather invest in our GPUs and in case they become dry with a lack of ML workloads, play CS: GO on them.

Conclusion

We finally got a static IP from Airtel and set up our systems. The final two cents are that it's fine to try out things up to a certain degree but know when to stop. Overall being in the discovery and MVP phase, we got our learnings, got to know about some new services out there and expand our horizons. But that might not be the case with you, so think.

If you're thinking about DDNS, don't just think about the cost of the static IP, think about the people time your workforce spent on it, because you pay them, it's costing you anyways.

Check out Atomic Asher and get in touch. We might be hiring, check our jobs page.

This blog is written by Wilfred Almeda, check out his other blogs too.

Disclaimer:

These are my thoughts, experiences and opinions. If you disagree with my points above, reach out to me and let's have a discussioin.

These thoughts do not reflect the opinions of Atomic Asher LLP.

Checkout Rusly
Checkout the Rusly GitHub repo

This blog lays out the system design thought process I used while designing and developing the system.

Read about the Rust syntactical development details here.

Why Rust?

I decided to use Rust due to the following reasons:

1. I wanted to implement my learning and make a project

2. I wanted to implement my system design knowledge

3. Rust promises features like fearless concurrency and memory safety. I wanted to see their viability when implementing production-grade systems.

API Endpoints

Let's take a quick look at the available endpoints

Why only 2 request params for /shorten?

I studied the UX of some popular URL shortener sites and decided to stick to the core purpose of just letting the user shorten the URL along with a custom URL for convenience.

What happens when a request to shorten a URL hits?

1. The URL validity is checked

2. If custom_url param is passed, its validity is checked

3. A random string of 7 alphabetic characters is generated

4. The string generated in step 3, the URL to shorten is stored in the database along with the URL to shorten and a UNIX timestamp

Database

Currently, the SQLite database is used as an embedded database.

Database Schema

Schema Description

id: The randomly generated string acts as the shortened URL string. This being a primary key ensures that there are no duplicate short URLs.

In the case of custom short URLs, it won't be allowed either.

Since the length of the short URL is 7 characters, there are over 8 billion possible combinations that are sufficient and uniqueness is maintained by the primary key.

fullUrl: The full URL string. Is fetched and the user is redirected permanently.

timestamp: The UNIX timestamp of entry. It is provided in the Rust code in the insert query. The commit time of the record and this timestamp may vary.

Why not store the full short URL?

To save storage space, only the short string is stored.

The host URL may change or vary and it'll cause complexities then.

Additional database update operations and compute will be required to update the fully stored short URL.

If this operation fails midway, data inconsistencies will arise.

By only storing the short URL string, a change in the host URL can be handled.

If the service is modified in the future, existing URLs can still be inculcated

Why SQLite?

SQLite is an excellent embedded database and considering the minimal database schema with only one table, it is a great choice.

It takes away the need for managing a dedicated database server. Which saves cost.

The following text is from the docs on when to use SQLite

Generally speaking, any site that gets fewer than 100K hits/day should work fine with SQLite. The 100K hits/day figure is a conservative estimate, not a hard upper bound. SQLite has been demonstrated to work with 10 times that amount of traffic.

SQLite will handle the load

Learn more about SQLite performance here.

Scaling SQLite

Potential Issues with Scaling

Replicated scaling of SQLite seems complex to me and can cause data inconsistencies and data synchronization issues.

If SQLite is set up in a replicated environment then each replica will have its own database and data copies.

If a /shorten write request is handled by replica A and subsequent read request for that data goes to the replica B then the user will face inconsistency issues, or the requests need to be served by the same replica which can overload a single replica.

Potential Solution for Scaling

If SQLite needs to be scaled then one possible approach is that the database file can be put on shared network storage and mounted in the replicas and the backend rust replicas can access it over the network.

This will increase response time as the file needs to be accessed over a network.

Each replica can have its own cache to avoid network database read calls.

But again, the caching database is yet another additional complex component to manage.

Understanding the random short URL string

The random short URL is a 7 lowercase characters alphabetic string.

Before this, I considered ULIDs and UUIDs to be used as short url strings but didn't proceed due to the following factors.

Both ULID and UUID are 128-bit long, the generated full-length string cannot be used as a short URL. If a substring is used then the computation spent on generating these full-length strings is wasted.

Both UUID and ULID strings consist of alphanumeric characters which didn't seem ideal for use as a short URL.

Personally, I feel there's nothing wrong with using an alphanumeric short string, but upon examining popular URL shorteners, I noticed that they don't use alphanumeric characters and thus decided to do the same. My best guess is that they do this from the perspective of user experience.

Existing methodology for generating random string.

Following is a function that generates alphabetic characters of a given length. It's Rust code but fairly easy to understand.

It is provided with a collection of alphabetic characters and it generates a random index and maps it to the equivalent character.

fn generate_shortened_url(length: usize) -> String {
    const CHARSET: &[u8] = b"abcdefghijklmnopqrstuvwxyz";

//Run the loop for n times
    (0..length)
       .map(|_| {
//generate a random number between 1 - charset.length
            let index = rand::thread_rng().gen_range(1..CHARSET.len());

//Pickout a character from the charset with the randomly generated index
            CHARSET[index] as char
        })
        .collect()
}

Performance: ULID/UUID vs Random String Generator

The performance of generating a 7-character random alphabetic string using the Rust function is faster than generating a UUID or a ULID.

This is because the function generates a random string by selecting characters from a pre-defined character set, which involves a simple operation of selecting a random index from the character set and converting it to a character.

In contrast, generating a UUID/ULID involves a more complex process of generating a random 128-bit value and encoding it in a specific format.

Current Deployment of Rusly

Rusly is currently deployed on Railway directly from its GitHub repo

Performance Metrics

Metrics are on their way. If you help with benchmarking then reach out to me.

Conclusion

Kudos to you that you read till the end. That's all for this blog.

I can go on and on about more system designing, reach out to me if you got something in mind.

Check out my other blogs at blog.wilfredalmeida.com/

Harbor is an open-source registry that secures artifacts with policies and role-based access control, ensures images are scanned and free from vulnerabilities, and signs images as trusted.

Here's a video I recorded for the same.

Device Hardware Specs (Recommended)

Before you begin the heavy lifting, you need some hardware juice. Following are some recommendations

Processor: 4 Core Intel i5/i7 10th gen plus or equivalent
RAM: 8GB workable, more is good
Storage: 40-60GB (Docker Images take up space)

If your device doesn't have at least the above specs, maybe try using a cloud VM or GitHub Codespaces or GitPod. There are some great free trials/referral bonuses on providers👨‍💻

Use these links to get started
Hetzner (Recommended, I use it. Cheap & Powerful)
DigitalOcean

Software Requirements

Following is a list of software with versions you need as of March 2023 for version 2.7.0. Check the official build guide to get the latest version details.

*optional, required if you use your own Golang environment.

Speaking of OS, you need a Linux-based OS. I tried WSL, but it didn't work, builds get stuck. You can use virtualization with VMware or Virtual Box if you're on Windows. Even Mac-OS works.

Test your Go & Docker installations before proceeding.

Forking & Cloning

Fork the official Harbor repository to your account.

Following is a snippet from the CONTRIBUTING.md file. Read the comments.

#Set golang environment
export GOPATH=$HOME/go # Add this to your .bashrc/equivalent file
mkdir -p $GOPATH/src/github.com/goharbor

#Get code
git clone --depth=1 git@github.com:goharbor/harbor.git # Add URL of your fork here
# Set the depth, you don't need everything
cd $GOPATH/src/github.com/goharbor/harbor

# The below steps can be performed later, you can skip if you want to

#Track repository under your personal account
git config push.default nothing # Anything to avoid pushing to goharbor/harbor by default
git remote rename origin goharbor
git remote add $USER git@github.com:$USER/harbor.git
git fetch $USER

Note: GOPATH can be any directory, the example above uses $HOME/go. Change $USER above to your own GitHub username.

Prerequisite YAML Config

Navigate to the $GOPATH/src/github.com/goharbor/harbor directory. To get Harbor running, first, a YAML config needs to be done in the make directory.

If you want to take a look at the codebase you can open it in VS Code or your favorite IDE/editor.

A sample YAML config file is provided as the make/harbor.yml.tmpl

Make a copy of this file or you can rename it as well, I'd recommend you make a copy.

The file should be named harbor.yml

Open the file in your favorite text editor.

The file looks something like this

Let's understand the config changes needed

• hostname: Line 5. Hostname to access admin UI. You can provide any non-conflicting URL here

• HTTP(S) ports: Lines 10 & 15. The HTTP ports that harbor will run on

• certificate & private_key: Lines 17, 18. The SSL certificate & key files

Following is how my config looks followed by an explanation

1. The hostname is a demo subdomain on my domain, it's not live. (Don't hit it hehe)

2. My development environment is a server VM so ports 80 & 443 are occupied, so I chose different ports.

3. The certificate & key paths. Their generation is explained in the following section.

Generating SSL Certificate & Key

To generate, you need OpenSSL installed. You can generate by any other way also, if you're clueless, just follow along.

Navigate to the directory you want the credentials to be generated and execute the following command. Replace demohub.wilfredalmeida.com with the hostname you specified above.

openssl req -x509 \
            -sha256 -days 356 \
            -nodes \
            -newkey rsa:2048 \
            -subj "/CN=demohub.wilfredalmeida.com/C=US/L=San Fransisco" \
            -keyout rootCA.key -out rootCA.crt

Two files named rootCA.key and rootCA.crt will be generated, specify their paths accordingly in the config and save and exit.

Building and Running Harbor

The bare minimum config needed to get Harbor up & running is now complete.

To run harbor, execute the following command in the project root.

make install

There are other options as well to run this, check out the compile guide

Note: This command will take time to execute. It takes ~25 - 30 mins for me, it depends on your hardware juice. Be patient.

If you're running on WSL, your build might get stuck at the following stage for which I have no solution

Once this build finishes, you'll see a message similar to the following

...
Start complete. You can visit harbor now.

Verifying Installation

If the build is completed successfully after the eternity it takes, you can load harbor UI in your browser by going to the HTTPS port on your localhost.

Here's an example https://localhost:2400. Note that the URL has HTTPS.

You might get a certificate warning in the browser due to the self-signed certificate, ignore that and proceed.

If Harbor loads in your browser then viola, you did it. Good Job!!🤝

Stopping Harbor

To stop harbor, run the following command. It'll stop all running containers for harbor.

make down

Testing code changes

If you want to make some code changes and check them, then run the following command

make versions_prepare compile_core  build -e BUILDTARGET="_build_core" -e PULL_BASE_FROM_DOCKERHUB=false prepare start

This command will compile the harbor-core image, build the necessary components for it and redeploy harbor.

If you're working on some other image like the harbor-db or harbor-portal, specify/replace it at the place of compile_core in the command.

It depends on your changes, however, the command takes ~2-3 mins to complete for me.

So in this comparatively little time, you can see your changes live

This command is courtesy of Wang Yan. It saves the hassle of executing make install

25 mins just to see 2 lines of code changes is not a pleasant experience.

Been there, done that. Not at all a pleasant experience🥹😆

That's all. Hope you're able to get Harbor up & running now.

For any queries, help join the Harbor Slack channels at the official CNCF Slack.

Reach out to me on Twitter or elsewhere from my portfolio.

Like my content? Check out my other blogs.

This blog is the final piece of the Custom CI/CD Pipeline series for my project ChaturMail: AI Email Generator📧

ArgoCD detects some change in a GitHub repo and triggers a new Kubernetes deployment.

Installing ArgoCD

We'll install ArgoCD on Kubernetes using one simple command

If you made it through the Hosting Harbor on VPS using NGINX as Reverse Proxy part then this is really easy😀

Before ArgoCD is installed, a namespace needs to be created, run the following command to create a namespace

kubectl create namespace argocd

Now that the namespace is created, ArgoCD can be finally installed, run the following command to install it

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

You'll see lots of stuff being created. Relax, it is all needed for ArgoCD to function properly.

Meanwhile, you can take a look at the YAML config in the link in the above command.

This will take some time for all the pods to set up and come up, check the status using the following command.

kubectl get pod -n

This command will list all existing pods and their status which will look something like this

This means that the pods are being created. So just wait.

While you wait, checkout my other blogs hehe🙃

Once all pods are up & running, the status will look something like this

Hooray!! ArgoCD is not installed

Accessing ArgoCD via Web GUI

ArgoCD is exposed via a service, to get all the services, run the following command

kubectl get svc -n argocd

This will list all available commands in the argocd namespace which will look something like this

The service argocd-server needs to be port forwarded to access ArgoCD. Run the following command to do so

kubectl port-forward -n argocd svc/argocd-server 8080:443

This command will set up a port forwarding from the port 443 of the argocd-server service to the port 8080 of our local host.

Now we can access ArgoCD at our localhost port 8080, load one of the following URLs in your browser

http://localhost:8080
http://127.0.0.1:8080

You will get a warning because the SSL certificate is self-signed, ignore the warning and proceed.

You'll land on the login page something like this

Here, the username is admin and password is an auto-generated secret. To get the password run the following command

kubectl -n argocd get secret argocd-initial-admin-secret -o yaml

This will output a password something like this. The password is base64 encoded and we need to decode it before it can be used

Run the following command to decode it, additionally, you can decode it anyhow you want you just need a base64 decoder

echo <super-secret-password-copied-from-above> | base64 --decode

Now the output is your password, keep it safe and login on to the web GUI, it'll look something like this

Deploying a project from GUI

Click on create a new app and then fill out all the necessary details like this. The important one here is SYNC POLICY which decides when to sync the changes.

• PRUNE RESOURCES: If you had some resource in your git repo and if you remove it, it'll be deleted from here as well.

• SELF HEAL: Changes via git repo are only allowed if you make any changes to pods using kubectl or anything else, they'll be reverted.

Now you need to provide the git repo to watch & the K8S YAML config to execute. Note that all YAML configs in the specified path will be applied.

Now we need to specify the Kubernetes namespace to deploy to

That's all, click Create and your app will be created and ArgoCD will check the given git repo every 3 minutes for changes. Once it detects any changes, it'll apply the YAML config from the provided path.

Deploying a project from CLI

You can also create an app from CLI, all you need is some YAML like the following. Explaining the directives is out of the scope of this blog, refer to the official docs.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-argo-application
  namespace: argocd
spec:
  project: default

  source:
    repoURL: https://github.com/WilfredAlmeida/MobXcess-Backend-Golang
    targetRevision: HEAD
    path: dev
  destination: 
    server: https://kubernetes.default.svc
    namespace: myapp

  syncPolicy:
    syncOptions:
    - CreateNamespace=true

    automated:
      selfHeal: true
      prune: true

To apply this config, you just need to run 1 command like this

kubectl apply -f myYaml.yaml

And voila!! that's all!!!

What's Next?

This blog concludes my Custom CI/CD Pipeline series for my project ChaturMail: AI Email Generator📧. Reach out to me on Twitter or anywhere else if you need any help. Check out my other blogs & follow for more.

This task is part of the Custom CI/CD Pipeline for the application ChaturMail: AI Email Generator

I was trying to store the NodeJS docker images on Docker Hub for Kubernetes to fetch but the upload speed to the images was around 500KBps and my image was over 500MB in size. Docker Hub also had other restrictions which led me to discover Harbor as an alternative.

Note: Kubernetes is referred to as K8S in a short form. Why 'K8S' because there are 8 characters between K & S.

Configuring & Installing Harbor

Getting the Helm Chart

Harbor is installed on k8s via a Helm chart by Bitnami. First, the Bitnami Helm repo needs to be added as follows

helm repo add bitnami https://charts.bitnami.com/bitnami

Now Harbor can be installed. Run the following command that'll get the YAML config. It is recommended to read the config and understand it before proceeding.

helm show values bitnami/harbor > harbor-values.yaml

This command will save the config in the file names harbor-values.yaml. It needs to be edited to set some configurations.

Helm chart configuration

Find and set the following parameters

• service.type: NodePort - The NGINX proxy service type. The default is set to LoadBalancer which overtook the NGINX of my VPS and my whole VPS became a Harbor service, which isn't desired. Harbor needs to be a k8s service on localhost. Read the docs to understand more

  This should look like this in your file

  

• externalURL: https://hub.example.com - The external URL for Harbor Core service. This is the URL the docker images will be tagged with

  

  .

• adminPassword: admin - The initial password of Harbor admin. Change it from the portal after launching Harbor

  

• commonName: hub.example.com - The common name used to generate the self-signed TLS certificates

  

That's all for the parameters. Now let's run the installation command

Installation

helm install harbor bitnami/harbor --values harbor-values.yml -n harbor --create-namespace

The above command will install harbor, be patient. Till it installs, let's understand the command

Understanding the installation command

helm install harbor bitnami/harbor: This indicates that we want to install harbor using the bitnami helm chart

--values harbor-values: This specifies the config values. The harbor-values.yml has edited config which will be used. If not specified, the helm chart with default values will be installed

-n harbor --create-namespace: Specifies the namespace in which harbor will be installed. It's 'harbor' in this case. The --create-namespace option will create the namespace if it doesn't exist already

To see the install progress, check the pod status

kubectl get pods -n harbor

Check the created services, a service with the name harbor will be created with an IP. This is the IP of the main harbor service. Note that the IP may vary.

Accessing installed harbor

If you load the IP in a browser or curl it, you'll get the harbor dashboard

Note: The IP is local, it won't load if you try to load your VPS's IP.

You might get an SSL warning from the browser. This is because the SSL certificate is self-signed and in the following section we'll make it trusted.

Enter the credentials set earlier and you'll be logged in to the dashboard.

Creating a new harbor project

Create a new project with the following config

Click on the created project and click on the 'PUSH COMMAND' option, it'll show sample commands to tag & push images

Making harbor available

Firstly, the commonName parameter set above needs to be mapped with the IP address of the service, to do so, it needs to be added to the /etc/hosts file. Open the file with your favorite editor and add the following line. Replace the IP address with the IP of your harbor service.

127.0.0.1    localhost
127.0.1.1    v2202210144615205508.goodsrv.de v2202210144615205508

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
89.58.55.191    v2202210144615205508.goodsrv.de v2202210144615205508

10.43.248.134 hub.example.com

Now, if you load/curl 'hub.example.com' then harbor will load. Note that it'll happen only on your VPS and not on the internet. You might see an SSL certificate error, this is because the certificate is self-signed. More on this is in the following section.

Trusting harbor certificate

The docker daemon needs to trust the certificate, to do this the /etc/docker/daemon.json needs to be edited, open it with your favorite editor and add the following config

{
    "insecure-registries":["hub.example.com"]
}

After saving this file, restart docker. If you have systemctl then use the following command

systemctl restart docker

Now if you try to push, docker will give an unauthorized error. To solve this, perform a docker login using the following command

docker login hub.example.com

Enter your credentials and you'll be logged in and now you can push images. Make sure your image is properly tagged.

Check your harbor dashboard and you'll find the image.

NGINX Config

Now you can use harbor. To access it via the internet, it needs to be exposed via NGINX. The following config will do it

server{

    listen 443 ssl;

    server_name hub.example.com;

    location / {

        proxy_pass https://hub.example.com;
        proxy_http_version 1.1;
        proxy_set_header   Host               $host:$server_port;
        proxy_set_header   X-Real-IP          $remote_addr;
        proxy_set_header   X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto  $scheme;

    }
}

The config might be needed to adjust as per your environment.

Note that the value given to the proxy_pass directive is the URL set in the /etc/hosts file. It doesn't point to a URL on the internet, it gets resolved locally.

Enabling SSL

My domain & VPS are mapped by Cloudflare, I use certbot. If you're using certbot, the following command will generate SSL given that the IP of your VPS is added as an A record in the DNS settings of your domain

certbot --nginx -d hub.example.com

Conclusion

That's all it takes to get harbor up and running. If you need any help/want to connect with me, reach out on Twitter.

Check out the pipeline series of blogs.

Check out ChaturMail: AI Email Generator

Keep your secrets safe :-)

The GitHub action does the following tasks:

1. Builds & Pushes Docker Image to Harbor

2. Update ArgoCD Config

tl;dr following is the complete YAML for the action followed by an explanation

name: BuildAndPushImageOnHarborAndUpdateArgoCDConfig

on:
  push:
    branches: [ "master" ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: docker/login-action@v1
      with:
        registry: harbor.example.com
        username: ${{ secrets.HARBOR_USERNAME  }}
        password: ${{ secrets.HARBOR_PASSWORD }}

    - uses: actions/checkout@v3
    - name: BuildAndPushImageOnHarbor
      run: |
        docker build ./ -t harbor.example.com/chaturmail/chaturmail-backend:${{ github.run_number }}
        docker push harbor.example.com/chaturmail/chaturmail-backend:${{ github.run_number }}

    - name: Clone Repository
      run: |
        git clone <argocd-config-repo-url>
    - name: Install yq
      run: |
        sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
        sudo chmod a+x /usr/local/bin/yq
    - name: Update YAML File
      run: |
        yq -i '.spec.template.spec.containers[0].image = "harbor.example.com/chaturmail/chaturmail-backend:${{ github.run_number }}"' 'argocd-configs/chaturmail-pod.yaml'

    - name: Push to Repo
      run: |
        git config --global user.name "${{secrets.USERNAME_GITHUB}}"
        git config --global user.email "${{secrets.EMAIL_GITHUB}}"
        cd argocd-test-configs
        git add .
        git commit -m "Updated by GitHub Actions"
        git push <argocd-config-repo-url> --all

Building and Pushing Docker Image to Harbor

Harbor is self-hosted on a VPS server and is exposed as a registry.

Logging in to Harbor

Login is done using the docker login action. The registry URI, username, and password need to be provided. Since Harbor is a docker registry the login works the same as for Docker Hub.

    - uses: docker/login-action@v1
      with:
        registry: harbor.example.com
        username: ${{ secrets.HARBOR_USERNAME  }}
        password: ${{ secrets.HARBOR_PASSWORD }}

Building & Pushing the Image

Shell commands are executed to update and push.

The image is tagged with the desired string which is of the format
<registry-uri>/<harbor-project-name>/<image-tag>:<github.run.number>

The github.run_number is added to keep the tags unique for each image.

Since the action runs on the master branch, the directory is specified as ./

    - uses: actions/checkout@v3
    - name: BuildAndPushImageOnHarbor
      run: |
        docker build ./ -t harbor.example.com/chaturmail/chaturmail-backend:${{ github.run_number }}
        docker push harbor.example.com/chaturmail/chaturmail-backend:${{ github.run_number }}

Updating ArgoCD Config

YAML config for ArgoCD to execute is maintained in a GitHub repo. ArgoCD watches the repo. Any changes to the master branch of the config repo trigger a deployment on the server.

The action updates the image name in the YAML config. ArgoCD detects this change and does a deployment using the config file with the new image name.

A shell utility yq is used to edit YAML.

    - name: Clone Repository
      run: |
        git clone <argocd-config-repo-url>

    - name: Install yq
      run: |
        sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
        sudo chmod a+x /usr/local/bin/yq

    - name: Update YAML File
      run: |
        yq -i '.spec.template.spec.containers[0].image = "harbor.example.com/chaturmail/chaturmail-backend:${{ github.run_number }}"' 'argocd-configs/chaturmail-pod.yaml'

Pushing ArgoCD config changes

Shell commands to login into GitHub, add changes to git staging, commit & push

    - name: Push to Repo
      run: |
        git config --global user.name "${{secrets.USERNAME_GITHUB}}"
        git config --global user.email "${{secrets.EMAIL_GITHUB}}"
        cd chaturmail-argocd-configs
        git add .
        git commit -m "Updated by GitHub Actions"
        git push <argocd-config-repo-url> --all

That's all. Check out the pipeline series of blogs.

Check out ChaturMail: AI Email Generator

Keep your secrets safe :-)

My portfolio and resume are hosted on my VPS, every time I made some changes in my resume latex, I had to manually login into the server and update the PDF.

To automate this, I set up a GitHub action that compiles my updated Latex into PDF and commits the PDF into the repo.

Now to auto-fetch the new PDF on the server, I wrote an Express API in TypeScript that takes a pull of the repo.

Following is its code

import express, {Application, Request, Response, NextFunction} from 'express';
import {resolve} from 'node:path';
import {execSync} from 'child_process';

//Express App
const app: Application = express();

//Webhook endpoint.
//Get's triggered by GitHub webhook
app.post('/',(req: Request,res: Response)=>{

    cloneRepo();

    res.send('Pulled the repo');
})

//Server listening on port 5000
app.listen(5000, ()=>{
    console.log("Server Running on port 5000");
})

//Function to execute the git pull shell command
const cloneRepo = ()=>{
    execSync('git pull origin main', {
        stdio: [0, 1, 2], 
        cwd: resolve(__dirname, '../../resume-files'),
      })
}

The API gets triggered by GitHub when a push is made to my repo and it executes a shell command that pulls the changes from the source repo.

The snippet can be customized to perform any action.

Started it in the background using PM2 and it works perfectly.

Note: I've used NGINX as a reverse proxy to expose the API to the internet.

Understanding Overall Pipeline Flow

1. Code changes are pushed to the master branch on GitHub

2. The repo has a Dockerfile. GitHub Actions does the following tasks

   • Build Docker Image

   • Push the image to Harbor Container Registry

   • Update YAML config being watched by ArgoCD

3. The Harbor Container Registry is hosted on my VPS, reverse proxied by NGINX, and domain mapped by Cloudflare. The GH action uploads the built image here

4. ArgoCD is installed on the VPS and watches a repo of YAML configs for Kubernetes. Any changes to the configs trigger a deployment by ArgoCD. The GH action updates the image tag in the YAML config

5. ArgoCD detects the change in YAML and initiates a K8S deployment

6. The docker image needed by K8S pods is fetched locally from Harbor

7. Kubernetes deploys and manages the pods w.r.t. to restarting, respawning, and load balancing

Role of NGINX

NGINX acts as a gateway to the VPS. All services are exposed to the internet utilizing NGINX. In the pipeline, NGINX plays the following roles:

• Reverse proxies Harbor

• Reverse proxies ChaturMail backend

Application Exposure

The backend system for ChaturMail is served by a Kubernetes load balancer service which gets an internal local-only IP address. This service is then reverse proxied by NGINX.

Role of Cloudflare

The services and applications exposed by the VPS are all configured to work with subdomains of my main domain wilfredalmeida.com

The domain is managed by Cloudflare and its IP address is proxied which prevents exposing of the original VPS IP and works in favor of Cloudflare's analytics and security services.

In the previous post, I discussed how to deploy Appwrite on Google Cloud. Now we'll see how to deploy on our Virtual Private Server (VPS).

VPS servers usually host a lot of services so we won't deploy Appwrite on the default ports HTTP(80) and HTTPS(443). If deployed on default, all services will go down and only Appwrite will run or the other services won't let Appwrite run on these ports.

Instead, we'll deploy it on localhost and serve it using NGINX as a reverse proxy.

Install Appwrite

First, we'll install Appwrite on our server. Appwrite comes as a set of Docker containers. So Docker needs to be installed mandatorily. Depending on your server OS, install Docker by referring to the docs.

Once docker is installed it takes only 1 command to get Appwrite installed.

PS: Don't paste the command and agree to everything by default. Some configs while installing need to be changed.

After running the command it'll pull a.k.a download some container images whose speed will depend on your internet speed so be patient.

sudo docker run -it --rm \
    --volume /var/run/docker.sock:/var/run/docker.sock \
    --volume "$(pwd)"/appwrite:/usr/src/code/appwrite:rw \
    --entrypoint="install" \
    appwrite/appwrite:1.1.1

Learn more about Appwrite installation from the docs.

While installing, Appwrite asks for HTTP and HTTPS ports for it to function which default to 80 and 443.

These need to be changed because if kept default all traffic sent to the VPS will be received by Appwrite and all other services hosted won't work.

So in the prompts, as shown below enter any ports from 1025 - 65535. Note that these ports need to be unique and should not conflict with any other ports to choose from accordingly.

PS: Don't choose common values like 3000, 8000, or 8080 because a lot of services default to these ports and will cause you problems later.

In our case let's choose ports 2021 -> HTTP and 2022 -> HTTPS. Make sure no other service is running on the ports you've chosen.

Note: Further on port 2021 is referred to because I've used it here. Make sure you're using the port you've specified.

Rest all values can be kept default for now but should be changed when in production. Learn more about them from the docs

Once all values are provided it'll take some time and Appwrite will be installed.

The console will load on the HTTP port specified above. To check if the console loads properly you can load it in the browser if your server has GUI or use curl as follows

curl http://localhost:2021/

If you see HTML code in your console then Appwrite has been installed successfully.

Expose by opening a port

Now to serve it over the internet you can either open the port of your server or use a Reverse Proxy. It might also be directly available on ip:2021 depending on the config of your server.

To open the port you need to allow it in your firewall/iptables. If you're using ufw then the following commands should work

sudo ufw allow 2021
sudo ufw reload

Now if you enter your server ip:port on your browser, Appwrite console will load. For eg. 172.0.0.45:2021

I'd recommend not opening any port and using a reverse proxy as it provides better control and security.

Reverse Proxy using NGINX

To serve using a reverse proxy you'd usually need a domain configured pointing to your server.

NGINX configs are stored on the directory /etc/nginx/sites-available. Here you can create a new config file or write to an existing config file.

Using your favorite editor open the file. Here we'll be creating a new file named appwrite using vi

sudo vi /etc/nginx/sites-available/appwrite

Write the NGINX config as follows

server{
    listen 80;
    server_name example.com www.example.com;

    location / {
        proxy_pass http://localhost:2021;
        proxy_http_version 1.1;
        proxy_set_header   Host               $host:$server_port;
        proxy_set_header   X-Real-IP          $remote_addr;
        proxy_set_header   X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto  $scheme;

    }
}

If you're facing problems while pasting in the editor or exiting it, please learn more about it :)

To exit the vi editor do the following steps:

• Press Escape key

• Type :wq

• Press Enter key

Verify the config is syntactically correct using the command

sudo nginx -t

Following is the command using systemctl

sudo systemctl restart nginx

That's all, Appwrite console will load on the domain you specified.

Note the proxy_pass http://localhost:2021; directive. It is the main reverse proxy config. All traffic received to the domain you specified will be sent to port 2021. That's where we have Appwrite running. Learn more about NGINX from the docs

If your domain URL is a subdomain, just enter it as the server_name value in the config. Also, note the way it's entered above, without http/s and any trailing /

Note: If your domain and server have SSL configured then the console might not load unless you configure SSL.

Configuring SSL

Depending on your server and domain, you'll need to handle SSL.

The following command will issue SSL using certbot. Learn more about certbot from the docs

sudo certbot --nginx -d example.com

Note: certbot needs port 80 of the server to be open, if it's closed, open it after running the command and close it again.

Following is the command to open using ufw

sudo ufw allow 80

On servers, for best security, it is recommended to always use SSL and keep port 80 closed. However, this depends on the services you're running.

That's it, now you have your own self-hosted Appwrite up and running using NGINX as a reverse proxy.

Appwrite is a great one-stop solution for various things. I personally like Appwrite because of the self-hosting feature it provides. All it takes is 1 docker command and you get a complete backend system up and running in minutes.

In this post, I've discussed how to deploy Appwrite on Google Cloud.

Creating a Compute Engine VM

1. Navigate to the Compute Engine Dashboard

1. Click on Create Instance

   

2. Creating Instance

   Appwrite docs recommend using a machine with at least 1 CPU core and 2GB of RAM.

   However, I tried using the f1-micro machine type from the N1 series which has 1vCPU and 614MB memory, but the deployment took over 25 minutes to get online so it's not recommended to use this machine type.

   I ended up using the n1-standard-1 machine type with 1vCPU and 3.75GB memory which at the time of writing costs me US$25.27/month. It took 5mins for the containers to initialize and get online after all config was provided.

   While creating the instance, there's an option of deploying a container image to the VM instance, I specified the Appwrite image from DockerHub but after the VM was created, the image wasn't deployed.

   

   tl;dr here's the shell command for creating the instance.

   PS: If it gives an error, you can try all config in the browser console and copy the command and paste it into the cloud shell. Just scroll to the end and you'll see a copy command option.

   gcloud compute instances create instance-1 --project={{PROJECT_ID_HERE}} --zone=us-central1-a --machine-type=n1-standard-1 --network-interface=network-tier=PREMIUM,subnet=default --maintenance-policy=MIGRATE --provisioning-model=STANDARD --service-account=230991838846-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --tags=http-server,https-server --create-disk=auto-delete=yes,boot=yes,device-name=instance-1,image=projects/debian-cloud/global/images/debian-11-bullseye-v20221102,mode=rw,size=10,type=projects/{{PROJECT_ID_HERE}}/zones/us-central1-a/diskTypes/pd-balanced --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --reservation-affinity=any
   

   Following is all the config I did for the VM

   

3. Connecting to the VM

   The VM after creation will get an external IP address that will be used to access Appwrite from the web. To install Appwrite, we need to SSH into the VM. Google Cloud provides SSH from the browser. The option to connect via SSH is present in the VM dashboard

   

   After clicking, a new window will open with the SSH connection established and a terminal view available.

   It is recommended to get updates using

   sudo apt-get update
   sudo apt-get upgrade
   

Installing and Configuring Appwrite

Now that the VM is created, Appwrite can be installed on it.

Appwrite installs as docker containers and docker is mandatorily required. The VM by default has Debian 11. Follow the official Docker docs to install docker. I installed it using the following command

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

Note: You might need to add the docker repository into apt. Refer to the docs.

Finally, once docker is installed, Appwrite can be installed. It takes only 1 command to install it. It is recommended to read the Appwrite installation docs. To install Appwrite, run the following command

sudo docker run -it --rm \
    --volume /var/run/docker.sock:/var/run/docker.sock \
    --volume "$(pwd)"/appwrite:/usr/src/code/appwrite:rw \
    --entrypoint="install" \
    appwrite/appwrite:1.1.1

While installing, some configuration options are asked, and I've kept everything default. Customize as per your needs.

Once the values are provided, it'll take some time and then Appwrite will be installed.

The Appwrite dashboard can be accessed using the external IP of the VM which can be found in the VM's dashboard on Google Cloud Console. Enter the IP in your browser and the console will load. A privacy warning might be shown due to the self-signed SSL certificate, it's fine, you can click Advanced->Proceed to unsafe and visit the console.

Appwrite installs an SSL certificate so all communication is secure and encrypted. The certificate is self-signed so your browser doesn't trust it so the warning is shown.

If you've changed the default ports for HTTP (80) and HTTPS (443) while installing then your console might not load.

That's all, you now have your self-hosted backend system up and running on Google Cloud.