<![CDATA[Agung Pratama Blog]]>http://127.0.0.1:8080/http://127.0.0.1:8080/favicon.pngAgung Pratama Bloghttp://127.0.0.1:8080/Ghost 5.69Fri, 20 Oct 2023 09:44:06 GMT60<![CDATA[Don't Forget to 'Media Rescan' on Android]]>I had this problem situation: I have this existing folder contains my photos, let's say /sdcard/september-photos, and that folder had been synced/backup to Google Photos. Then after a while, I transferred new photos to the folder via adb tools -- tried Android File Manager on macbook,

]]>
http://127.0.0.1:8080/post/2023-10-05-dont-forget-media-rescan-on-android/652d58b34598a10001ab9cd5Thu, 05 Oct 2023 07:20:45 GMT

I had this problem situation: I have this existing folder contains my photos, let's say /sdcard/september-photos, and that folder had been synced/backup to Google Photos. Then after a while, I transferred new photos to the folder via adb tools -- tried Android File Manager on macbook, but somehow it failed to connect to my pixel, hence using adb manually --, I checked that it transferred succesfully, but Google Photos didn't show the new photos.

After a quick googling, I found out that in android, you need to do Media Rescan to update the "internal database" contains the index of all media files (e.g: photo, music, video, etc), especially if you add the files by using adb script. However, I didn't find this problem if I copied the files directly via android file manager (e.g: external sdcard is connected via usb-c and i transferred the files via android's file manager), seems like the file manager automatically does that for you.

Tried this adb command solution:

adb shell am broadcast -a android.intent.action.MEDIA_MOUNTED -d file:///sdcard

But it got me this error

Broadcasting: Intent { act=android.intent.action.MEDIA_MOUNTED dat=file:///... flg=0x400000 }

Exception occurred while executing 'broadcast':
java.lang.SecurityException: Permission Denial: not allowed to send broadcast android.intent.action.MEDIA_MOUNTED from pid=10757, uid=2000
	at com.android.server.am.ActivityManagerService.broadcastIntentLocked(ActivityManagerService.java:13783)
	at com.android.server.am.ActivityManagerService.broadcastIntentLocked(ActivityManagerService.java:13623)
	at com.android.server.am.ActivityManagerService.broadcastIntentWithFeature(ActivityManagerService.java:14501)
	at com.android.server.am.ActivityManagerShellCommand.runSendBroadcast(ActivityManagerShellCommand.java:808)
	at com.android.server.am.ActivityManagerShellCommand.onCommand(ActivityManagerShellCommand.java:221)
	at com.android.modules.utils.BasicShellCommandHandler.exec(BasicShellCommandHandler.java:97)
	at android.os.ShellCommand.exec(ShellCommand.java:38)
	at com.android.server.am.ActivityManagerService.onShellCommand(ActivityManagerService.java:9249)
	at android.os.Binder.shellCommand(Binder.java:1049)
	at android.os.Binder.onTransact(Binder.java:877)
	at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:4731)
	at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2628)
	at android.os.Binder.execTransactInternal(Binder.java:1285)
	at android.os.Binder.execTransact(Binder.java:1244)

Until now, I haven't found the solution using the adb command above (if you know, please let me know in the comment).

Fortunately, there is this android app called mediaReScan:updateMediaStorage, I finally able to refresh the media databases on my folder /sdcard/september-photos. It lets you chose the folder(s) to be scanned and updated the android's "internal database".

Hope this is useful!

]]>
<![CDATA[The Android's Google Photos App Won't Open .CR3 File]]>I have Canon M50 and I shoot RAW. And it saves the raw in CR3 format. When I copied the CR3 files to my pixel and tried to open the folder on Google Photos, it doesn't show load the images at all.

Looking on this page, there is

]]>
http://127.0.0.1:8080/post/2023-10-04-the-androids-google-photos-app-wont-open-cr3-file/652d58b34598a10001ab9cd1Wed, 04 Oct 2023 08:49:59 GMT

I have Canon M50 and I shoot RAW. And it saves the raw in CR3 format. When I copied the CR3 files to my pixel and tried to open the folder on Google Photos, it doesn't show load the images at all.

Looking on this page, there is this statement:

Making it easier to manage, share and edit your original photos (including CR3 files)* in your Google Photos library

And the written footnote:

*Offer expires and must be redeemed by May 31, 2021. Available to new Google One members only. Terms Apply.

I am using Google One subscription, but still my CR3 image files can't be opened. So in my desperate move, I renamed CR3 to CR2 extension. And voila, it works!!!! 😂

So, the only thing I need to do is to rename all those CR3 files (more than 100 images). After trying some file manager, it seems in android, it is impossible to the extension file on multiple files at once -- please let me know if you have good app that can do that. And in macbook, it is also impossible to do this using the Finder :(

So my last resort (to rename all CR3 files to CR2 extension) is by using this simple oneliner script:

# Note: this only works for filename that doesn't have multiple '.' (dot) character. 
# E.g: 
# - IMG-1111.CR3 -> works
# - IMG.AA.CR3 -> doesn't work

ls *.CR3 | cut -d'.' -f1 | xargs -I@  mv @.CR3 @.CR2

Enjoy!

]]>
<![CDATA[How to SSH to Private Instance via Jumphost/Bastion]]>

There is frequent cases where we want to provision instance in a private network for security reason (e.g: database instance), but when we don't have Corporate VPN to connect to that private network, how we can configure this?

The answer is via a jumphost or some people

]]>
http://127.0.0.1:8080/post/2021-11-16-how-to-use-jumphost-bastion-to-ssh-to-instance-in-private-subnet/652d58b34598a10001ab9ccdTue, 16 Nov 2021 03:58:07 GMT

There is frequent cases where we want to provision instance in a private network for security reason (e.g: database instance), but when we don't have Corporate VPN to connect to that private network, how we can configure this?

The answer is via a jumphost or some people call it bastion host. Basically it's a intermediary instance provisioned in the public network, that only open ssh port to the internet (and secured via private-public ssh key).

Let say we have 2 instances jumphost and db (the private instance), we can ssh to the private instance by using this config :

Host jumphost
  HostName <jumphost-public-ip>
  User ubuntu
  AddKeysToAgent yes

  IdentityFile <private-key-file-location>

Host db
  Hostname <db-private-ip>
  User ubuntu
  Port 22
  IdentityFile <private-key-file-location>
  ProxyCommand ssh -q -W %h:%p jumphost

Then to connect into db, we can just ssh db.

]]>
<![CDATA[Recover Corrupt ZSH History File]]>

Very often I got this error when opening the terminal session (normally after the computer wasn't shutdown properly):

zsh: corrupt history file /home/agung/.zsh_history

There is a simple fix for that, thanks to this article. Just invoke this:

#!/usr/bin/env zsh
# George Ornbo (shapeshed) http:
]]>
http://127.0.0.1:8080/post/2021-07-29-recover-corrupt-zsh-history-file/652d58b34598a10001ab9cc9Sat, 06 Mar 2021 09:31:20 GMT

Very often I got this error when opening the terminal session (normally after the computer wasn't shutdown properly):

zsh: corrupt history file /home/agung/.zsh_history

There is a simple fix for that, thanks to this article. Just invoke this:

#!/usr/bin/env zsh
# George Ornbo (shapeshed) http://shapeshed.com
# License - http://unlicense.org
#
# Fixes a corrupt .zsh_history file

mv ~/.zsh_history ~/.zsh_history_bad
strings -eS ~/.zsh_history_bad > ~/.zsh_history
fc -R ~/.zsh_history
rm ~/.zsh_history_bad
]]>
<![CDATA[Assign Fixed Name to The Network Interface on Raspbian]]>I had this problem when adding a new usb wifi dongle for my raspberry setup. That it sometimes gets assigned with wlan0 name. I expect that wlan0 name should be assigned to the built-in wifi card device, hence the wifi dongle should get wlan1 name instead. But that's

]]>
http://127.0.0.1:8080/post/2020-02-14-assign-interface-name-to-device/652d58b34598a10001ab9cc5Fri, 14 Feb 2020 00:00:00 GMT

I had this problem when adding a new usb wifi dongle for my raspberry setup. That it sometimes gets assigned with wlan0 name. I expect that wlan0 name should be assigned to the built-in wifi card device, hence the wifi dongle should get wlan1 name instead. But that's not the case. This leads to problem, since I need this naming to be deterministic for my network configuration (e.g: which device acts as wifi client vs which one as access point, which one does support 5 Ghz band vs only support 2.4 GHz band).

Fortunately there are ways to solve this.

Predictable Network Interface Names

The straightforward way to enable this is via raspi-config. This feature provides a predictable naming mechanism by generating the name from device firmware/topology/location information[1].

Assigning fixed names based on firmware/topology/location information has the big advantage that the names are fully automatic, fully predictable, that they stay fixed even if hardware is added or removed (i.e. no reenumeration takes place) and that broken hardware can be replaced seamlessly. That said, they admittedly are sometimes harder to read than the "eth0" or "wlan0" everybody is used to. Example: "enp5s0"

In the Network Options, enable predictable network device name, and reboot.
Assign Fixed Name to The Network Interface on Raspbian

This solution is simple and easy. However, the assigned name (something like this wlp3s0 enp0s31f6) are harder to remember compared to the old naming wlan0, wlan1, and eth0. Which brings me to seek another simple solution that can achieve this classic naming.

Since raspbian uses systemd for the init system, we can utilize the systemd.link[2]. Basically what we can do with it is to put a simple rule that assigns a certain name for device with certain mac address. So I can just take a note of my wifi device mac address and assign it a unique name.

Since I have 2 wifi devices, I setup two .link files:

> cat /etc/systemd/network/09-wlan1.link
[Match]
MACAddress=d0:37:45:5a:c7:22

[Link]
Name=wlan1

> cat /etc/systemd/network/10-wlan0.link  
[Match]
MACAddress=dc:a6:32:4b:1e:d7

[Link]
Name=wlan0

That's it and reboot.

References


  1. Predictable Network Interface Names ↩︎

  2. systemd.link — Network device configuration ↩︎

]]>
<![CDATA[How to Backup and Restore Your Raspberry Pi Image]]>I have been using my raspberry pi for almost a month. During this time, I have configured a lot on it: installing required softwares and tools for work, configuring the network and hotspot, etc.

Considering the amount of time and effort spent, it is very important for me to backup

]]>
http://127.0.0.1:8080/post/2020-02-05-how-to-backup-and-clone-your-raspberry-pi-image/652d58b24598a10001ab9cc1Wed, 05 Feb 2020 10:47:16 GMT

I have been using my raspberry pi for almost a month. During this time, I have configured a lot on it: installing required softwares and tools for work, configuring the network and hotspot, etc.

Considering the amount of time and effort spent, it is very important for me to backup the raspberry. In fact, the first day I received my raspberry pi, after having setup so many things, I accidentally disconnect the power cable causing my raspberry didn't boot at all.

All my effort were vanished 😱

How to Backup and Restore Your Raspberry Pi Image

After that incident, I quickly researched how to take a snapshot, clone it on another microsd card and successfully booted from it.

So here we go, the step by step using Debian/Ubuntu:

Image Backup

Put the microsd on computer. It will be mounted by default in the /media/<user>/.

> ls /media/agung/
boot  rootfs

Please keep note of boot and rootfs, as this is the raspberry partitions name.

Now invoke this to see where the partition disk is located

> df -h
Filesystem      Size  Used Avail Use% Mounted on
....
/dev/sdc1       253M   54M  199M  22% /media/agung/boot
/dev/sdc2        29G   11G   18G  38% /media/agung/rootfs
....

As you can see, in my case it is located at /dev/sdc1 and /dev/sdc2, so the disk is /dev/sdc. Please keep note that yours may be different. The number after sdc is basically saying about partition number.

Now comes to the snapshot step. Please prepare available disk size more than the size of your microsd card. So if you are using 16GB microsd, eventhough the content of the raspberry pi is less than that, ensure that you have 16 GB of free room to keep the backup on the computer.

sudo dd bs=4M if=/dev/sdc of=/home/agung/Backups/raspberry-pi-2020-02-05.img

Above command use dd to do low-level copy from the /dev/sdc to the backup location indicated by of argument value. Please note you should use absolute path for of. Also in Mac OS, you should use bs=4m.

Please wait for a while as it can take some times. You can track the progress by this command:

watch du -sh /home/agung/Backups/raspberry-pi-2020-02-05.img 

Resize Snapshot Image

As you noticed, the snapshot size is equal to the size of microsd capacity. We can make the snapshot smaller.

First, we need to mount the snapshot image. To do so, first we need to find the unused loop device:

> sudo losetup -f
/dev/loop20

Then we tell it to link the loop device from the snapshot image

sudo losetup -P /dev/loop20 /home/agung/Backups/raspberry-pi-2020-02-05.img

Now at this stage, the /dev/loop20 already linked to the snapshot image. The -P flag is there so kernel is force scan the partition table on the disk.

Next, we can now use gparted to modify the partition.

If you don't have gparted installed, just install it by sudo apt install gparted.

sudo gparted /dev/loop20

It will show list of partitions inside the disk:

How to Backup and Restore Your Raspberry Pi Image

The /dev/loop20p1 is the first partition and the /dev/loop20p2 is the second. So they are boot and rootfs partition that we saw in the backup section.

Now we need to reduce the partition size of /dev/loop20p2 (rootfs partition).

  • Right click on /dev/loop20p2 and choose Resize/Move
  • Move the black right arrow from right to left to resize the partition. The white part is the unused part (contains no data) while the yellow has data.

How to Backup and Restore Your Raspberry Pi Image

Just to be safe, I added more free storage (white part) around 1-2 GB.

How to Backup and Restore Your Raspberry Pi Image

  • Click Apply (the green checkmark) to apply the resize operation.Now to check if the filesystem is okay:

So we resized the partition, however the snapshot image size is still not reduced. There are a few things to do, to free up the unused space:

  • Calculate the partition size using fdisk and input p as the command:
  • The sector size is 512 bytes and the end sector is 27807743. Since sector number is started at 0, then we need to truncate the snapshot with this:

Done, by now the snapshot image size should be reduced.

Restore/Clone Image from Snapshot

Now with the snapshot image that we have, we can restore this snapshot to the microsd card with this step:

sudo dd if=/home/agung/Backups/raspberry-pi-2020-02-05.img of=/dev/sdc 

Yes, we just reverse the input (if) and output (of) location.

That's it 😃

Bonus: Reclaim More Storage on The Restored MicroSD Card

Since we restore the microsd card from the size-reduced snapshot image, then consequently the disk partition size is also not utilizing the full capacity of the sd card too.

You can reclaim the storage by:

sudo raspi-config

Then choose Advanced Options -> Expand Filesystem

How to Backup and Restore Your Raspberry Pi Image

References

]]>
<![CDATA[Some references for setup raspberry]]>

Useful references:

Setup access point: https://www.raspberrypi.org/documentation/configuration/wireless/access-point.md
stop at https://www.raspberrypi.org/documentation/configuration/wireless/access-point.md#internet-sharing

fixing wlan0

]]>
http://127.0.0.1:8080/post/2020-01-31-some-references-for-setup-raspberry/652d58b24598a10001ab9cbdFri, 31 Jan 2020 13:53:56 GMT

Useful references:

Setup access point: https://www.raspberrypi.org/documentation/configuration/wireless/access-point.md
stop at https://www.raspberrypi.org/documentation/configuration/wireless/access-point.md#internet-sharing

fixing wlan0 gets assigned to wifi dongle
https://www.raspberrypi.org/forums/viewtopic.php?p=756998#p756998

]]>
<![CDATA[How to Assign Static IP to Network Interface uap0 RaspAP]]>So, recently I encountered intermittent problem when running the RaspAP on my raspberry pi. The problem is on the interface uap0, created by RaspAP for Hotspot access point. This interface supposed to have static ip 192.168.50.1.

However, sometimes it gets assigned with entirely different ip range (e.

]]>
http://127.0.0.1:8080/post/2020-01-29-how-to-assign-static-ip-to-network-interface-uap0-raspap/652d58b24598a10001ab9cb9Wed, 29 Jan 2020 11:58:03 GMT

So, recently I encountered intermittent problem when running the RaspAP on my raspberry pi. The problem is on the interface uap0, created by RaspAP for Hotspot access point. This interface supposed to have static ip 192.168.50.1.

However, sometimes it gets assigned with entirely different ip range (e.g: 168.x.x.x) when I messed up with RaspAP's setting.

Although I am still looking at the root cause of this issue (it is very hard to reproduce), I already have mitigation solution to remedy this problem:

Assign uap0 static ip using ifconfig

sudo ifconfig uap0 192.168.50.1 netmask 255.255.255.0 up

Then restart the raspap-service

sudo systemctl restart raspap.service

Assign uap0 ip via RaspAP Web Interface

The other way than using terminal access above is using the RaspAP web interface. Just point your browser to the raspberry's RaspAP url. On Configure Networking menu, select uap0 tab then configure like below:

How to Assign Static IP to Network Interface uap0 RaspAP

Set empty for the rest of configuration (e.g: default gateway, dns, etc).

Then click Save Setting and Apply Setting.

References:

]]>
<![CDATA[Troubleshooting Cheatsheet when Debugging Service in Kubernetes]]>

Gain Shell Access to Pod (similar to ssh)

kubectl exec -it <pod> -- bash

This only working if the pod contains bash. Change bash to sh if bash not found.

Temporary Spawn Ubuntu Pod

Sometimes it is impossible to gain shell access to the existing running pod. The

]]>
http://127.0.0.1:8080/post/2020-01-21-troubleshooting-cheatsheet-when-debugging-service-in-kubernetes/652d58b24598a10001ab9cb5Tue, 21 Jan 2020 07:21:40 GMT

Gain Shell Access to Pod (similar to ssh)

kubectl exec -it <pod> -- bash

This only working if the pod contains bash. Change bash to sh if bash not found.

Temporary Spawn Ubuntu Pod

Sometimes it is impossible to gain shell access to the existing running pod. The reason maybe: no shell binary in the pod or even we don't have no access to the pod.
The other way that we can try is spawning a pod temporarily then we can use this pod to debug the running service.

kubectl run -i \
	--tty busybox \
	--image=ubuntu \
	--restart=Never \
	-- bash

In that pod, you can use apt to install common utilities to debug service:

apt update -y
apt install -y dnsutils curl netcat

After finish, don't forget to delete the pod

kubectl delete pod busybox

Useful Commands

# check the dns resolution (you can try to lookup kubernetes service host as well here)
nslookup <domain>

# check whether the host:port is open
nc -zv <host> <port> 
]]>
<![CDATA[Moving The Blog to The New Domain]]>Yesterday I got a notification email saying my blog domain: agung.io will be expired soon. I planned to renew, however the price is quite hefty: 60 USD (around IDR 900k), double in price than my first year of renting this domain.

Since I want to keep my blog setup

]]>
http://127.0.0.1:8080/post/2020-01-19-moving-the-blog-to-the-new-domain/652d58b24598a10001ab9cb1Sun, 19 Jan 2020 02:54:19 GMT

Yesterday I got a notification email saying my blog domain: agung.io will be expired soon. I planned to renew, however the price is quite hefty: 60 USD (around IDR 900k), double in price than my first year of renting this domain.

Since I want to keep my blog setup cost as low as possible without sacrificing many things, I excercised how easy it is to move my blog to a new domain. So, I purchased new cheap domain agung.tech with only USD 3 for the first year in GoDaddy. There are 2 outcome from my experiment of migrating this blog to new domain:

  • either it is really painful and error prone process, hence I would keep paying the 60 buck to continue using agung.io domain, or
  • it is relatively easy, so it is better to use the new agung.tech domain for this blog

Result

It was super easy and quick. In fact, I managed to migrate this in 1-2 hours. The migration process basically involved few configuration changes. It also worth to mention that my blog content doesn't have any hardcoding of the blog domain to link resources (e.g: image resources).

The Migration Process

To give you some context, I use:

  • GitHub to put my blog content, using hugo
  • Netlify to host the blog. Domain agung.io is also managed within the Netlify.
  • New domain agung.tech is purchased on GoDaddy.

Transfer DNS Control from GoDaddy to Netlify

Just to be clear, the main reason why I delegate the DNS control to Netlify is to keep my life simple. Rather than managing the DNS from GoDaddy, but using Netlify to manage the blog, I'd instead choose Netlify for both.

The easy way (I would say the only way, but CMIIW) to do this is by configure the NS record of agung.tech in GoDaddy to Netlify's nameservers.

Note that this can take some times to propagate. To check it:

dig agung.tech -t NS

When it is already reflected, in the ANSWER SECTION it should print out the new nameservers.

;; ANSWER SECTION:
agung.tech.             3600    IN      NS      dns1.p04.nsone.net.
agung.tech.             3600    IN      NS      dns2.p04.nsone.net.
agung.tech.             3600    IN      NS      dns3.p04.nsone.net.
agung.tech.             3600    IN      NS      dns4.p04.nsone.net.

Then, I added agung.tech in Netlify domain setting so I can start configuring the DNS there.

Points blog.agung.tech to My Netlify Blog Domain

By default, Netlify will give your blog the netlify.com domain. For instance, my blog domain is agung.netlify.com.

So I added CNAME record of blog.agung.tech to agung.netlify.com. This basically means that agung.netlify.com is the canonical name of blog.agung.tech, hence will return the same IP location

Configure Netlify Custom Domain

In order for Netlify able to route the new domain to my site, I added blog.agung.tech in the Custom Domain list. Even I set it as Primary Domain.

Change Hugo BaseURL

In the config.toml, it was set to

baseURL = "https://blog.agung.io/"

I changed to

baseURL = "https://blog.agung.tech/"

Now, I can access my blog via the new domain.

However, there is one more step to properly routing someone who access my blog via old domain to the new domain.

Redirect Old Domain to New Domain

Beside setting the old domain CNAME to blog.agung.tech, I also modified Netlify _redirect file

https://agung.netlify.com/* https://blog.agung.tech/:splat 301!
https://blog.agung.io/* https://blog.agung.tech/:splat 301!

Basically that will redirect agung.netlify.com or blog.agung.io to my new domain.

That's it. Quite easy right :D

References:

]]>
<![CDATA[Mocking gRPC in Go]]>

I found this useful article on how to do gRPC mocking in Go.

This gRPC mocking will be super useful to setup testing of your golang application that has dependency to another service invoked via gRPC.

I put it here just so I can revisit this later and try it

]]>
http://127.0.0.1:8080/post/2019-08-04-mocking-grpc-in-go/652d58b24598a10001ab9cadSun, 04 Aug 2019 01:02:44 GMT

I found this useful article on how to do gRPC mocking in Go.

This gRPC mocking will be super useful to setup testing of your golang application that has dependency to another service invoked via gRPC.

I put it here just so I can revisit this later and try it by myself.

]]>
<![CDATA[My Blog Setup - Re-Evaluate Blog Setup]]>If you have read my post about my blog framework consideration, then you may know that I chose:

  • Hugo: for my blog framework, one of popular Static Site Generator frameworks out there.
  • Gitlab Page: for hosting my static content.
  • Gitlab CI: for automate my blog deployment to Gitlab Page.

My

]]>
http://127.0.0.1:8080/post/2019-03-06-my-blog-reevaluate-blog-setup/652d58b24598a10001ab9ca9Wed, 06 Mar 2019 12:27:17 GMT

If you have read my post about my blog framework consideration, then you may know that I chose:

  • Hugo: for my blog framework, one of popular Static Site Generator frameworks out there.
  • Gitlab Page: for hosting my static content.
  • Gitlab CI: for automate my blog deployment to Gitlab Page.

My quote from that post:

Choosing Hugo as my blog content engine comes to 1 consequence: only Gitlab Page support Hugo with Gitlab CI to automate the process of publishing the website. On the other hand, GitHub Page only supports Jekkyl.

Actually I can use both, but I have to spend more effort on the CICD thing when using GitHub and no seamless experience on that yet. Gitlab basically offer the simplicity by providing Hugo Blog Template and Hugo CICD template, that's a win for me 💛 (at least in the beginning)

Yup, I have to say using Gitlab Page and Gitlab CI is really simple and very straightforward for a beginner like me. It helped me to focus on learning Hugo system for a while at beginning, at least for the first 2 hours since I had started to bootstrap my blog then.

In that first 2 hours, I also tried figuring out how my ideal blog writing and blog tuning workflow could be. Note that this workflow is thought by assuming that the blog author is only myself.

This was what I had thought due to my blog system setup at that moment: for each 1 single task (such as: add new content, fixing comment integration section, etc):

  1. Create a new branch from master
  2. Make a change
  3. Hit hugo serve in local to check how it looks
  4. Fix it and try again, go to point 3 until satisfied, then
  5. Merge it with master
  6. Push it to origin Gitlab, then Gitlab CI will take care of it

But then I realized the existing workflow using that system, has some downsides:

  • Can't live preview online on a certain branch. This is a major downside for me. The gaps that it can be filled:

    • Final test before going live to production blog. It is necessary and very practical, since testing my blog locally doesn't catch all integration aspects. Example: Disqus comment section and social sharing feature aren't working well in local mode. Even there are bug occurrences in my content that doesn't get caught at local-test but somehow broken when going live. This is mostly due to me messing up with hyperlink and base URL path stuff.
    • Very convenient for sharing with friends for proofreading.
  • The workflow above makes me rely on my laptop to do blogging. This is actually not a major complaint, but nice to have solution for this.

    • Very often, when I was away quite a long time (e.g: travel, stroll around), I draft my new blog post on my mobile phone. To post through via mobile phone is almost impossible. Actually you can: by using Gitlab online editor. But that's not really good interface. Besides, uploading the image and refer it to content post is harder in mobile (believe me, I've tried it, it made me irritated). The workflow requires me to have access on my terminal: to use git command, hit hugo serve to check locally.

So the next 1 hour, I had been frantically trying to search for "easy and simple" solution to address those weak points and finally found it :smirk:

Online Site Preview from PR Branch

Gitlab Page and Github Page able to show the content from 1 specific branch only (default is master branch). This is their weakness at this moment.

Fortunately, I found there is Netlify. What is it? Quoted from their website:

Build, deploy, and manage modern web projects

An all-in-one workflow that combines global deployment, continuous integration, and automatic HTTPS. And that’s just the beginning.

I thought this could be my solution. Reading more about Netlify:

  • It offers seamless integration with GitHub/Gitlab/and other major git hosting provider. So I can still keep using Git as my blog storage
  • Out of the box supports for most Static Generator Framework, Hugo is one of them. It automatically configures the CICD step for Hugo.
  • It has Deploy Preview feature which I can set to be triggered whenever there is a new commit on active PR branch. Yup, this point is what I was looking for.
  • It also has a nice notification feature, such as whenever the blog deployment is finished, it notifies me via email.
My Blog Setup - Re-Evaluate Blog Setup

The Deploy Preview is a great feature, I can preview my PR branch online and test it. This is accessible through random URL generated by them (e.g: https://xxxxxxx-.netlify.com).

I succeed migrating my blog from Gitlab Page to Netlify without breaking too much sweat, thanks to Netlify great integration with most Git hosting provider + Hugo. I even invested my money to rent my own agung.io domain through Netlify. Actually, they offered quite a nice discount for my own domain + free HTTPS/SSL certificate for my website. Thanks, Netlify!

CMS for Admin

CMS or Content Management System is the answer to negate the second weakness point: to reduce my dependence on a laptop. Normally with CMS, you want some basic features offered: able to draft new content, upload image, and preview it. Having a WYSIWYG editor is a plus point. All those tasks should be accomplished through the web interface at least since smartphone mobile browser is quite powerful nowadays.

There is only 1 free and open source solution (that I know of) that offers CMS feature on top of Hugo, it's Netlify CMS. Yup, turns out Netlify has open source CMS and it has great Hugo integration as well. I guess it was a good decision to choose Hugo. And now I am starting to see why I also love Netlify ❤️.

CMS User Management

I had successfully set it up and my blog's CMS Admin interface is protected using Netlify Identity feature.

My Blog Setup - Re-Evaluate Blog Setup My Blog Setup - Re-Evaluate Blog Setup

CMS x Git Hosting

In order for Netlify CMS fully working, I had to configure my Netlify hosting (where my blog and CMS Admin are running) to use Git Gateway so the CMS can read and write my blog's repository in Gitlab.

CMS Editorial Workflow

Now, there is called Editorial Workflow in Netlify CMS Admin, which a web workflow so that editor can add/edit a post and set the status to: Draft, In Review, and Ready before publishing it. Under the hood, whenever creating a draft (edit post, create a new post), it creates a new branch from master and open pull request. Also, to set the draft status, it uses low-level git metadata.

Here's the thing: Editorial Workflow is only supported by GitHub at this moment 😓. But that's okay since I could easily migrate my blog from Gitlab to GitHub and I did it 😄.

CMS Dashboard

My Blog Setup - Re-Evaluate Blog Setup My Blog Setup - Re-Evaluate Blog Setup My Blog Setup - Re-Evaluate Blog Setup

There is one minor point of CMS Admin dashboard though. With my iPhone 7 Plus screen, I have to change to landscape mode to show some invisible buttons (e.g: image upload button, delete image button, etc).

Summary

So, finally, after those 2-3 hours of effort, I've changed my blog from using Gitlab + Gitlab Page + Gitlab CI to GitHub + Netlify (Hosting + CICD). My blog writing workflow now is improved to my wants, and so far I've been using it for 2 weeks and it has been a pleasure blog writing experience for me. The workflow then is like below:

  1. Create a new branch from master
  2. Make a change
  3. Hit hugo serve in local to check how it looks
  4. Fix it and try again, go to point 3 until satisfied, then:
  5. If the changes are complex: push to remote GitHub and open Pull Request and see Netlify's Deploy Preview; commit; push; until satisfied.
  6. Merge with master branch and Netlify will take care production blog deployment.

If you have some facts and opinion about Netlify, please feel free to comment below.

Thanks for reading my post!

References:

]]>
<![CDATA[My Blog Setup - Hugo How to Create New Post]]>

In previous post, I have explained some important setting and configuration on Hugo site so you can personalize it to your own like.

Create New Post and Page

Now comes the important part of the blog itself, the content, writing the content and publish it.

Basically there are 2 categories

]]>
http://127.0.0.1:8080/post/2019-03-06-my-blog-hugo-how-to-create-post/652d58b14598a10001ab9ca5Wed, 06 Mar 2019 11:17:52 GMT

In previous post, I have explained some important setting and configuration on Hugo site so you can personalize it to your own like.

Create New Post and Page

Now comes the important part of the blog itself, the content, writing the content and publish it.

Basically there are 2 categories of content: Post and Page as explained in content directory section of my previous post:

  • page
    This is a placeholder where you can put all independent page that doesn't show like the post entry. For example: About Us, Contact Us, and Portfolio Page.
  • post
    All content in this folder will appear descendingly by post's published date. This will be your main folder where you put the site's posts.

Create new post and page is similar.

To create a new post, put it under post folder:

hugo new content/post/<title>.md

And to create a new page, put it under page folder:

hugo new content/page/<title>.md

To test add new post:

hugo new content/post/this-is-my-first-post.md

The new file is created and hugo will bootstrapped the content with this:

---
title: "This Is My First Post"
author: ""
type: ""
date: 2019-03-06T17:56:22+07:00
subtitle: ""
image: ""
tags: []
---

Now the part between 2 --- (e.g: title to tags) is called frontmatter. TL;DR: it's the metadata about the post/page. I will explain about that later.

To add the content, just write it under the second --- part. Example:

---
title: "This Is My First Post"
author: ""
type: ""
date: 2019-03-06T17:56:22+07:00
subtitle: ""
image: ""
tags: []
---

Hi all, 

This is my first post.

Try to save that and invoke hugo serve to access it locally. You should see the post right?

You can continue to write the post, git commit and push to remote (gitlab). The gitlab CI will take care the deployment of your latest blog writing 😄.

Frontmatter, what is it?

At the top of the post, it has what it's called frontmatter. It is basically a post/page metadata and Hugo (and the theme) can show that to the audience. Some themes even introduces their exclusive metadata. Beautifulhugo theme for example, it has bigimg field and process that information to show the big image on top of the post.

There is also some magic on hugo new command, it automatically infer the metadata field (e.g: title and date in this case), right?. hugo new can fill them because it uses the current theme's archetypes/default.md content.

themes/beautifulhugo/archetypes
└── default.md

The themes/beautifulhugo/archetypes/default.md:

---
title: "{{ replace .Name "-" " " | title }}"
author: ""
type: ""
date: {{ .Date }}
subtitle: ""
image: ""
tags: []
---

As you can see, it has {{ and }} syntax. It's basically the template syntax and evaluated by hugo command. You can read more about it here.

  • title: "{{ replace .Name "-" " " | title }}", means it is filled by .Name variable. That surely comes from the filename without .md extension, when you create the post `hugo new content/post/this-is-my-first-post.md
  • date: {{ .Date }}, it is filled with current date.

What's Next

Next post I will tell you my opinion and some reflections after using the Hugo, Gitlab Page and Gitlab CI for my blog.

]]>
<![CDATA[My Blog Setup - First Look at Hugo Configuration Setting]]>On the previous post, I have explained how to bootstrap the blog repository and its CICD and publish it to the public. The site is accessible from https://<user>.gitlab.io/<repo-name>, in case of mine: https://bangau1.gitlab.io/my-blog/.

A bit of disclaimer, that

]]>
http://127.0.0.1:8080/post/2019-03-04-my-blog-setup-first-look-at-hugo-configuration-setting/652d58b14598a10001ab9ca1Mon, 04 Mar 2019 07:57:04 GMT

On the previous post, I have explained how to bootstrap the blog repository and its CICD and publish it to the public. The site is accessible from https://<user>.gitlab.io/<repo-name>, in case of mine: https://bangau1.gitlab.io/my-blog/.

A bit of disclaimer, that is not my real blog repository. Mine is private :smile:

Hugo Directory and Content Structure

My Blog Setup - First Look at Hugo Configuration Setting

Disclaimer: I only explain the content structure of Hugo, the Hugo Template by Gitlab.

Now, before we take a look at some important configuration in Hugo, let me explain in general how the Hugo content structure

├── config.toml
├── content
│   ├── _index.md
│   ├── page
│   │   └── about.md
│   └── post
│       ├── 2015-01-04-first-post.md
│       ├── 2015-01-15-pirates.md
├── resources
│   └── _gen
│       ├── assets
│       └── images
├── static
│   └── favicon.ico
└── themes
    └── beautifulhugo

config.toml file

This is the Hugo configuration file in toml format that contains all settings about:

  • information about the site: Author information and Social Link (e.g: Facebook, Twitter, GitHub, Gitlab, etc)
  • the site system and content rendering parameter

You can use another Hugo supported format: yaml and json.

content directory

This is the content location. It consists of 2 subdirectories:

  • page
    This is a placeholder where you can put all independent page that doesn't show like the post entry. For example: About Us, Contact Us, and Portfolio Page.
  • post
    All content in this folder will appear descendingly by post's published date. This will be your main folder where you put the site's posts.

It also has a special file _index.md. This is for to display a sticky content Post entry. Let say you have a lot of posts and the reader will go through some pagination. This sticky content will be always displayed. It may be useful in some cases: such as Important Announcement to the readers.

resources directory

TBH, I don't know much about this folder other than it is used by Hugo to generate the assets and images after doing some image processing. You can basically ignore this at this time.

static directory

In contrast to resources folder, in static directory you can put pretty much any static files in there. Normally you put Site's icon asset, the avatar, and other assets that you can think of.

themes

Pretty obvious, it's to include the theme for the site. From Gitlab Hugo Template, by default, it uses beautifulhugo theme.

Hugo Important Configurations

Take a look at config.toml file, this is the initial configuration generated by Gitlab Hugo template:

baseurl = "https://bangau1.gitlab.io/my-blog/"
contentdir    = "content"
layoutdir     = "layouts"
publishdir    = "public"
title = "Beautiful Hugo"
canonifyurls  = true

DefaultContentLanguage = "en"
theme = "beautifulhugo"
metaDataFormat = "yaml"
pygmentsUseClasses = true
pygmentCodeFences = true
#disqusShortname = "XXX"
#googleAnalytics = "XXX"

[Params]
  subtitle = "Hugo Blog Template for GitLab Pages"
  logo = "img/avatar-icon.png"
  favicon = "img/favicon.ico"
  dateFormat = "January 2, 2006"
  commit = false
  rss = true
  comments = true
#  gcse = "012345678901234567890:abcdefghijk" # Get your code from google.com/cse. Make sure to go to "Look and Feel" and change Layout to "Full Width" and Theme to "Classic"

#[[Params.bigimg]]
#  src = "img/triangle.jpg"
#  desc = "Triangle"
#[[Params.bigimg]]
#  src = "img/sphere.jpg"
#  desc = "Sphere"
#[[Params.bigimg]]
#  src = "img/hexagon.jpg"
#  desc = "Hexagon"

[Author]
  name = "Some Person"
  email = "youremail@domain.com"
  facebook = "username"
  googleplus = "+username" # or xxxxxxxxxxxxxxxxxxxxx
  gitlab = "username"
  github = "username"
  twitter = "username"
  reddit = "username"
  linkedin = "username"
  xing = "username"
  stackoverflow = "users/XXXXXXX/username"
  snapchat = "username"
  instagram = "username"
  youtube = "user/username" # or channel/channelname
  soundcloud = "username"
  spotify = "username"
  bandcamp = "username"
  itchio = "username"
  keybase = "username"


[[menu.main]]
    name = "Blog"
    url = ""
    weight = 1

[[menu.main]]
    name = "About"
    url = "page/about/"
    weight = 3

[[menu.main]]
    identifier = "samples"
    name = "Samples"
    weight = 2

[[menu.main]]
    parent = "samples"
    name = "Big Image Sample"
    url = "post/2017-03-07-bigimg-sample"
    weight = 1

[[menu.main]]
    parent = "samples"
    name = "Math Sample"
    url = "post/2017-03-05-math-sample"
    weight = 2

[[menu.main]]
    parent = "samples"
    name = "Code Sample"
    url = "post/2016-03-08-code-sample"
    weight = 3

[[menu.main]]
    name = "Tags"
    url = "tags"
    weight = 3

I won't explain all of them, but only the important things to make your blog truly yours (not from template anymore)

baseurl

This must be set correctly so Hugo can render the asset links correctly by using this baseurl information. Please set it to the public domain and path of your site.

title

The title of your blog site.

disquessShortname

This is commented by default. Please uncomment it so that you can use Disqus to provide reader able to comment on your blog post. Go to Disqus Homepage, sign up, retrieve the short name, and put it in config.toml file.

googleAnalytics

Hugo doesn't offer stats view analytics tool out of the box. However, you can use Google Analytics (GA) to gather knowledge about your blog stats view. Put your GA user id there, usually in the format UA-xxxxxxx-xx.

enableemoji

This is not displayed by default in the configuration file. But since emoji in our daily life has become a thing, I think using emoji in the blog would be nice added-value too. Just put enableemoji = true in config.toml.

Params.subtitle

The subtitle under [Param] section is for the subtitle of the blog.

Params.logo and Params.favicon

These settings are important. Put the customized logo and favicon in the static folder and refer it to the configuration.
For example, this is my blog configuration snippet

[Params]
  subtitle = "Movie lover. Engineer. DevOps"
  logo = "avatar.jpg"
  favicon = "avatar.jpg"

And I put the avatar.jpg both for logo and favicon in static folder

blog.agung.io/static
├── avatar.jpg

Params.Author

This is where you put all the information about the blog author and her/his social media link account. If you don't use some of them, you can just remove them from the configuration.

Params.menu.main

This is a special parameter to organize the top bar menu of the blog. Here you can specify multiple menus, name them, put it under the parent menu if you want, and sorted them by the weight (low weight means the most left).
Example for this blog, I only have these 3 menus

[[menu.main]]
    name = "Blog"
    url = ""
    weight = 1

[[menu.main]]
    name = "About"
    url = "page/about/"
    weight = 2

[[menu.main]]
    name = "Tags"
    url = "tags"
    weight = 3

I think I've listed all necessary configuration to make the blog more personalized.

If you think there are other additional configurations that are important, please comment below :smile:

Test in Local

Now to play around with these configurations and contents without having to publish it first, you can test it locally first.

  1. Install Hugo binary that you can find the instruction here
  2. Go to the blog repository folder, then invoke
hugo serve

That command will build the blog and print information where you can access it locally. Normally it's http://localhost:1313

Pretty simple and fun right?

What's Next

Next post, I am gonna tell you the workflow on creating a new blog post with Hugo. Stay tuned!

]]>
<![CDATA[My Blog Setup - Bootstrap Blog Repository and CI/CD]]>As explained in my previous blog post, I have to chose Gitlab Page as it offers seamless experience on setting up my preferred Static Site Generator: Hugo and the necessary CI/CD. The CI/CD is necessary here since the public site is in HTML format, but my blog content

]]>
http://127.0.0.1:8080/post/2019-03-04-my-blog-setup-bootstrap-blog-repo-and-ci-cd/652d58b14598a10001ab9c9dSun, 03 Mar 2019 16:58:59 GMTAs explained in my previous blog post, I have to chose Gitlab Page as it offers seamless experience on setting up my preferred Static Site Generator: Hugo and the necessary CI/CD. The CI/CD is necessary here since the public site is in HTML format, but my blog content will be in a mix of Markdown, HTML, and some templating format stuff. The CI/CD is using Gitlab CI, invoking specific Hugo command to generate all my content to HTML format.

Now, about Gitlab Page itself, this is a feature offering by Gitlab, so everyone can put static content and publish it publicly accessible from:

  • https://<user>.gitlab.io/<repo-name>, if you chose the Project Site, or
  • https://<user>.gitlab.io, if you chose the User or Organisation Site

I chose the first option due to the fact I maybe in future will introduce another Gitlab Page and I don’t want to mess my another blog if I chose the second option, since it has common subdomain and prefix.

The public content can be set from certain branch, I chose the default: master branch.

Okay, that’s enough for the intermezzo part. So here’s how easy it is to setup Gitlab Page

  1. Create Gitlab account

  2. Select New Project, and chose Create from Template and chose Pages/Hugo

    Create From Template

  3. Set the Project Name to my-blog. Note that it’s not my blog setup, I chose another project name, and I won’t disclose that information 😛

    Project setup

    You can set the repository as private if you want. Then hit “Create” repo button

  4. Now let’s see how it goes

    The site is accessible from https://<user>.gitlab.com/my-blog. In this case, mine is https://bangau1.gitlab.com/my-blog. At the time I write this post, I get 404 not found error.

    404 error not found

    This is expected by the way, since there’s one specific configuration that needs to be done, that is to change the baseurl in the config.toml file from the template. Change it to the site url, in my case it’s https://bangau1.gitlab.io/my-blog/.

    setup base url

    Normally the CICD pipeline will be triggered and here’s mine

    Gitlab CI Dashboard

    Once the CICD is finished executing, the site is accessible

    Finally it's shown

That’s it, easy right? What do you think about the process overall? Feel free to add any comment/suggestion below :pray:

Next post, I will explain some important Hugo configuration settings, (e.g: change the avatar and favicon, the frontmatter in Hugo, etc) and how to test it on local environment.

]]>