I think ... - url-shortener

Create a free go-link server “on edge” using Cloudflare Worker KV

2021-06-06T00:00:00+06:00

Among quite a few ways to implement a go-link server (i.e., url-forwarder, short-url server, etc.), I’m going to show you how to use free-tier Cloudflare Worker (& KV) to create an in-house, on-edge, no-webserver go-link server.

For example, the short-link for this article is go.kmonsoor.com/golink-kv

/latest (by which I mean go.yourdomain.co/latest) may point to https://www.yourcompany.com/about/news which is a public page
/hr-help may point to https://www.company-internal.com/long-link/hr/contact.html, which is company’s internal human-resources help portal
/cnypromo may point to https://shop.yourcompany.com/sales/promotions/?marketing-promo=2021-cny which is a temporary sales promotions page targeting the shoppers during the Chinese new year of 2021.

Please note that using the setup and the code below, it’ll be possible to resolve short-links via a single sub-domain, e.g., go.your-domain.co. However, it’s possible (with some modification of the code) to resolve/redirect via any number of domains (your own, of course) towards any other public or private URL, and all sorts of novelties. However, for brevity’s sake, I will discuss the first one, a single sub-domain usecase.

To set up a go-link server or short-URL resolver via a proper KV+Worker combination, we’ll go through these steps:

Pre-requisites
Create the short-link map as a KV
Mapping a KV to a Worker variable
Handling a route with webworker
Create the Worker
Pointing a DNS record to the Worker
Next step
Related

Pre-requisites¶

The DNS resolver for the root domain (in the example below, kmonsoor.com) needs to be Cloudflare. Because the core of the solution, the “worker”, runs on the nearest (from the user) edge of Cloudflare using a standard KV (“key, value”) list.
Write permission to the DNS configuration as you’d need to add a new AAAA DNS record.
Some knowledge of Javascript(ES6), as we are going to write the “worker” in that language.

Create the short-link map as a KV¶

We’ll start the setup by creating the short-link map, the list between the short-link segments that you (or someone in your org) define, and the actual URLs they need to point to.

Find the KV stuff in the Workers section. From the screenshot, please ignore the “Route” section for now.

you’d need to create a Worker KV “Namespace”. Name the namespace as you seem fit. I named it REDIRECTS (in all caps just as a convention, not required).
List the short links & their respective target URLs. From the examples in the intro, the keys latest, hr-help, cnypromo etc. would be in as the “key”, and the target full links as the respective “value”.
Remember NOT to start the short part with ‘/’. It’ll be taken care of in the code.

Once you’ve listed all your desired (short-link, target-link) combinations, now we have a KV on Cloudflare. However, it’s not referencable from your Worker code, not yet. Hence the next step.

Mapping a KV to a Worker variable¶

Now, we will map the previously created KV to a variable that can be referenced from our Worker code. Please note that though I used different names, it can be the same as well. Also, note that multiple Workers can access a single KV, and vice versa is also true; a single Worker can reference multiple KVs.

Handling a route with webworker¶

Create the Worker¶

Now, we will write Worker-code that runs on V8 runtime on the nearest (from the requesting user) “edge” location of Cloudflare, to execute the code and deliver the result(s) to the user. In this case, that would be to redirect user-requested address to the mapped one (by you, in the KV namespace above).

The code editor looks like this:

If you rather prefer to copy-paste, please feel free to do it from the below GitHub Gist.

Once done, it should look like …

Pointing a DNS record to the Worker¶

Finally, we need to point a DNS record that’ll redirect all requests to your re-soutign sub-domain (e.g. go.your-domain.com) to the Cloudflare Worker that we just created.

According to the Cloudflare docs, the DNNS record must be an AAAA record, pointing to the IPv6 address 100::. The “Name” here is the “sub-domain” part of your choice, which is better be short, to rightfully serve our goal here.

Voila ! Now, test some of the short-urls that you’ve mapped via the KV. Enjoy ! Watch out for the target usage though. Here’s the limit.

I think you’ll be fine, unless you’re some celebrity ;)

Next step¶

As the next step, I’m thinking to create a generic Go/Link resolver browser extension. Then, someone can set their own default domain or company domain of choice as short-domain host. In that case, entering just go/hr-help on the browser will take to https://www.company-internal.com/.../hr/contact.html that we have discussed at the beginning (remember the example case of an internal human resources help portal?).

If you want to do this url-direction on your server, but only using webserver, try this: Personal short-link server using only Caddyserver

If you find this post helpful, you can show your support through Patreon or by buying me a coffee. Thanks!

Deploying a short-link aka go-link server using only Caddyserver

2021-04-16T00:00:00+06:00

Before I go into any details, please note that I’ve used the term “shortlink-server” instead of “url-shortener” because the difference is significant for this post. A url-shortener takes a long url, and gives a short url, then redirects any requests for the shortened link to its longer counterpart. On the contrary, shortlink-server takes both the long and short url as inputs, then only does the redirect-ion part.

Backstory¶

If I wanted a fantastic, personal url-shortener or “go link” server, there are many excellent solutions out there, which are not only free or open-source but full-fledged as well for personal or public usage. Instead, I wanted some “service” that would “resolve” my personal, short links. I have been using bit.ly for a long time for its customizable “short-half” part, but the problem with bit.ly is – for some God-forsaken reason – blocked by the Bangladeshi govt. So, I needed a replacement to be appropriately “glocal”. There are many free (unbranded) and commercial (branded) options as well, but I wanted something that would be resolved via my hosted service (and personal domain) and as cheap as possible. So, basically solution for a poor nerd :D

Before jumped into this solution, I tried (deployed & tested) few others myself, mainly kellegous/go, kutt.it and adamyi/golinks. But, all of them “too featureful” for my needs.

What I wanted is to be able to:

resolve only my custom shortlinks (hence, no need for url-shortener)
not a public, internet-facing service (hence, any frontend, authentication, email verification etc. would be overkill )
minimal setup (if possible, no webapp at all)

Given my previous experience with Caddy webserver, which is an amazing one(why?), I had a gut feeling that Caddy has something for me – under the sleeve – to meet my minimal set of requirements. Thankfully, I managed to find it.

I believe NGINX, currently the most popular webserver, has some kind of similar mechanism as well. But, I’m not an expert, and once I was genuinely intimidated by its config file syntax. YMMV.

What you gonna need?¶

your own domain which will be the root of the shortlinks. While sub-domained URL like go.company-name.com/* is quite common, if you have some short domain, like you.co/*, only for this purpose, that’s fine as well.
A web-host server or public-facing instance with its own, public IPv4 address.
working knowledge of Linux

Step-1: Point your subdomain to the right place¶

Find out what’s the puplic IPv4 address of your instance that’ll act as the webserver. It’s usually on the cloud management dashboard.
make sure that, regardless of your cloud architecture (e.g. VPC, subnet, firewall etc.), the SSL port (:443) of the instance is reachable from the public internet.
now go to your domain name registrar (or, DNS management provider which in my case is Cloudflare). There, you need to point shortlink subdomain (go.)to the webserver’s I.P. address. In DNS terms, you’ll be creating a CNAME entry on the domain’s nameserver table.

You can do this step as the last one. But for some reason, I prefer it to do first. Because sometimes, DNS propagation takes some time. But, once my web service is up and running, I like to see the result instantaneously. ;)

Step-2: Install Caddy, a mighty webserver¶

Depending on your host OS (Ubuntu 20.04 LTS in my case), you need to install the Caddy webserver. While there are some hacky solutions to run, I think running Caddy as a background service is the simplest to manage. In fact, the documentation of Caddy is excellent, so I’d better leave that part to you.

After running with the default config(Caddyfile), (in Ubuntu’s case, located as /etc/caddy/Caddyfile), it should show a status somewhat like the below image. Please note that, in many cases, if running without sudo, Caddy cannot attach itself with the SSL port (:443), which is necessary for serving https://. So, check for that error message in the “status” log.

PS: By the way, want your console and command prompt to look 🚀 like mine? Here’s the guide: How do I pimp up my terminal on Linux

Step-3: Tell Caddy your short-links to redirect¶

Now, it’s time to configure Caddy to actually do the job.

Caddy has its native redir “directive” to redirect incoming web-request from one to another. While the map directive is relatively new, it makes the config file, i.e., Caddyfile, look elegant in case you have (or will have in the long run) a long list of short-links.

Here’s mine, which is working nicely …

# /etc/caddy/Caddyfile

go.kmonsoor.com {   # replace it your web-url root

    map {path} {redirect-uri} {
        /blog    https://blog.kmonsoor.com
        /photos  https://photos.kmonsoor.com

        /resume     https://drive.google.com/file/d/1nMS3i1ai6nsI70zZ7NFnNQ_XmvAa4GOl
        /resume-doc https://docs.google.com/document/d/1ECx1Yr8Jzz9I3S5VcoKnZQz56oIht2XaM5gSNetcWag

        /rickrolled    https://www.youtube.com/watch?v=dQw4w9WgXcQ

        # will add new ones here like the above
        # ...
    }

    # this below code is required to actually make the above `map` work

    @hasRedir expression `{redirect-uri} != ""`
    redir @hasRedir {redirect-uri}

    # code below is to set the default response if the requested shortlink isn't here
    respond "Thas's an unknown short URL ... :("  
}

Note: Don’t forget to restart the caddy service to let the new config to take effect.

Step-4: Profit¶

Yeah, that’s it. Now, add some own personal stuff with some cool short-links, and proudly share with the world.

What’s next ?¶

I’m thinking that given the very low workload my shortlink resolver needs — unless I’m becoming an overnight internet sensation — using a server instance only for this purpose is overkill. My next goal is to have the same service using some “serverless” function or using the “worker on the edge” thing from Cloudflare. Let’s see ;)

Update Now, actually done it. Here is the link: Free short-link server “on edge” using Cloudflare Worker K.V.

If you find this post helpful, you can show your support through Patreon or Paypal or by buying me a coffee. Thanks!