What is Nextcloud?
What Nextcloud is not!
Benefits and comparison
Who it is for?
- If you work in a profession that you generate a lot of data.
- if you live in a different part of the world than the actual country you came from that you have to move from place to place. (like me)
- If you have to constantly make the data you generate available with your partners in a remote location.
- If you want to share photographs, home videos, or music with your distributed family.
- Single board computer enthusiasts wanting to learn more and make the best use case of investment.
What you need to setup self hosted nextcloud?
- Raspberry Pi 3/4 (4GB or above)
or any Single board computer(sbc) (5-12 volt) you can keep up online that you do not waste electricity bill. Since I have much different make sbc, so I decided to go with a multi-node cluster, but you can do it on a Virtual Machine on a Windows laptop too. I am using a Raspberry Pi 3 and Raspberry Pi 4 on a remote site i.e. back home in India. The reason I suggest raspberry pi is because it is arm64 architecture, it has a large user base, and above all, it's Made in the UK.
- Storage of 1TB SSD or HDD (more the merrier)
- I am using a 1 TB NVMe SSD here in London. And a 1TB standard HDD in India.
Ethernet Cables CAT 7 should be sufficient.
I do not recommend connecting your pi to the router over wifi.
- USB to MicroSD card Adapter
- Ubuntu Operating System
- ProxySQL (for load balancing Galera Cluster)
- Syncthing (for copying data to other site and high availability)
- CoreDNS (for DNS Service at home network)
- Haproxy (for load balancing Redis and Apache2)
- Redis (to manage file locking and php session sharing)
- Latest Version of Nextcloud
Prepare your first Node
Install Web Server
Once that download completes, unzip the file with the command:
Now Lets configure Apache
Note, I setup nextcloud on my.cloud.com/nextcloud and not directly on my.cloud.com. This is as I wanted to put something else on web root.
Save and close the file. Enable the new site with the command:
Installing and configuring php
Installing and configure mariadb
After this run the command below for you ready to use.
Log in to the MySQL prompt with the command:
Configure mysql/mariadb for Nextcloud
Finish up the database with the following commands:
Finally, restart Apache with the command:
Completing the first node Nextcloud installation
Now our first node setup is complete.
Take a note of your raspberry pi's IP using the ifconfig command.
Let's assume your pi is connected to the router using the ethernet and the eth0 interface IP assigned is 192.168.0.2. In your router setting configure your raspberry pi with a static up address so that it never changes to a different IP address.
Let's first set up OpenVPN, it's super easy.
Method 1. Using Openvpn Server to access over wan.
Secondly, it will say if you are using static IP or not, depending on what you have you can choose but in case you do not have static IP then enter the domain name you registered (in our case its my.cloud.com)
Copy this file to your phone or another computer where you would need to install the OpenVPN client. Browse this file and enter the username password to connect. Now you can open nextcloud using your raspberry pi as if you are at home. Note: To be able to access nextcloud using the registered domain you need to add it to /var/www/html/nextloud/config/config.php trusted domain array. In our case, we add my.cloud.com.
Method 2: Port forward 443, to access over https://yourdomain.com
Dry Run to check if everything is fine setup
Finally the last step for single node nextcloud instance
Save and quit, then enable the new configuration:
First step to multisite
Install DNS service on Site 1 Nodes 1 & 2 : CoreDNS
And copy Corefile from my github account to /etc/coredns/. You would have to create coredns folder manually. Take a note of this line in the Corefile
Set CoreDNS IP on your router's LAN > Primary DNS Server IP, the secondary DNS server IP can be 18.104.22.168 or 22.214.171.124 i.e. either google or Cloudflare or any resolver of your choice.
Secondly on all your server node's /etc/resolv.conf set two lines of both the coredns server such as
Redis configuration for Sentinel setup
Sentinel acts as a configuration provider or a source of authority for clients service discovery.
Furthermore, Sentinel is a robust distributed system, where multiple sentinels need to agree to about the fact a given master is no longer available. Then only the failover process starts a select a new MASTER node. This sentinel agreement is done according to the quorum value.
What is Quorum ?
Let’s get our hands dirty with Redis Sentinel.
You can either choose to set up supervised by systemd in the config or create a systemd unit under /etc/systemd/system/
Setting up Mariadb Galera Cluster
Initially, I set it up using Rsync Snapshot State Transfer but I quickly learned that using the rsync method halts the frontend clients while the wsrep state transfer is in progress. I re-setup the Galera cluster using Xtrabackup-v2 as the SST method.
On an important Note: Set the data directory to the path on your SSD to gain performance. otherwise nextcloud will be very slow.
Follow the link to my GitHub account to copy the galera configuration on each of these nodes to /etc/mysql/conf.d/galera.conf later (Do not start Galera cluster as of yet, put galera.conf on desktop temporarily). After you confirm that the Nextcloud instance has been copied to both the nodes. Do not forget to update the trusted domain array in config.php.
I have saved the full nextcloud config on my Github account which you can use as a reference.
Do not forget to set slaveof in redis.conf node 2 and node 3 and server id in mariadb configuration.
Setup the syncs between nodes and both Sites
Take a look at this link to an intro and how to setup syncs
Install and Configure ProxySQL Load Balancer for Mariadb
PS: I highly recommend don't waste your time and efforts on using clustercontrol it only looks fancy but later you would have to bear the pain. I would say it is only meant for large infrastructures. I quickly got rid of it.
To set the administrator password in ProxySQL, we’ll connect to that configuration database and update the appropriate variables.
First, access the administration interface. You’ll be prompted for a password which, on a default installation, is admin.
- memory, which is altered when making modifications from the command-line interface.
- runtime, which is used by ProxySQL as the effective configuration.
- disk, which is used to make a configuration persist across restarts.
Connecting 2 sites database server galera clusters
We achieve this by doing my setting up Master to Master Replication between two nodes of MariaDB servers of each site.
Instead of specifying another site MariaDB IP address, you specify the IP address and port of the server running ProxySQL. (Normally in master to master replication we connect two instances by specifying their node IPs but here we specify ProxySQL) ProxySQL further takes the request to the MariaDB Galera cluster which replicates the information to each Galera node.
Then on node 1 of Site B in the MySQL client run:
Setup HAProxy to load balance web servers
Although load balancing Galera cluster too can be done through HAProxy, ProxySQL gave me fine control. With HAProxy we will load balance Apache2 and Redis Sentinel. Configuring HAProxy is quite easy.