Great video once more, really admirable how you are able to bring this complex material in an easy to understand way 👏🏻 Looking forward to your kubernetes set-up, as I think it would be great if I could just add more resources to these type of containers when needed. Indeed, bridging would also be nice, because it is hard to have others switch to matrix. Did you also considers dendrite instead of synapse? Mastodon would also be very interesting. Your channel is a true gem 💎!
Thanks so much, Robert. I chose Matrix simply because it is the largest, and most mature from the research I did, and my own experience (even though there are others that have similar features). I find if you're trying to convince people to jump ship it's easier when it's something that is "widely" used. I will likely do a follow up on a Discord bridge as I'm going to set that up for my own Jim's Garage Discord server anyway. Kubernetes is coming, there's just a few more 'core' apps and items I want to cover first. I know kubernetes is going to be niche, so I'm keen to have people fully set up on docker first.
echo the request for bridges - I also use telegram with bots for my internal alerting so bridging would be great. I'd also agree a different piece of content would be a Mastodon server and maybe how to integrate with Matirx (I'm assuming they can as they both seem federated but maybe I'm just deluded 🙂
A major problem isn't technical - it's about getting your contacts to use alternatives other than the usual suspects. That perhaps is more of a challenge than actually installing and using the software.
I agree with you, adoption and mind switching is the hardest part. Hopefully I've called out how seamless it should be for people, but it's still a bit more effort than just downloading something 'that works'. Hopefully the benefits are in line with the effort.
I also could not get in touch through matrix, but thank you. This video helped me setup matrix synapse and I managed to figure out the proxy and whatsapp bridge myself. Now I can finally read messages from WhatsApp groups on my iPhone without having to install that Meta stuff which i refuse to use. Sadly most people don’t care about privacy.
Good and clear explanation. I've been running Synapse for probably a year or so now as "one chat to rule them all"--I have the bridges installed for iMessage, Signal, Telegram, and WhatsApp, so I can use one app to communicate with any of those. Including iMessage on a Windows machine--that bridge is kind of fiddly to set up (and it looks like they're in the process of a complete overhaul of it), but it still works pretty well.
@@Jims-Garage That wasn't really my purpose in installing it, so I haven't pursued having my non-techie friends/family use my Synapse server. Most of them are using my mail server, which is a start, and a few are using my self-hosted Bitwarden server. Once thing at a time... I have it installed on an Oracle VPS, which is free since it's small enough. There's an Ansible installation that takes care of most of the details (except, at the time, for the iMessage bridge)--Synapse itself, Synapse Admin, web client(s) of your choice, bridges of choice, etc. Pretty comprehensive, but it's a lot of moving pieces.
Nice solid video for getting yourself set up with a basic Matrix instance. Well done! I've been running a server for a group of friends for just over two years now, it's been absolutely rock steady and has been surprisingly simple to keep maintained. Slowly over time I'm bringing more friends on-board. It runs really smooth on an on-prem server with UPS backup and redundant failover networking, so has about 99.8% uptime so far (downtime is me tinkering with it lol). Tried to drop you a message via the room link, but seems it's not there anymore.
Hi Jim. Thanks, cracking video - I'm busy working through it to get Matrix installed. I have a couple of questions regarding the initial docker command at timecode 11:50 and onwards: Why does it have to be run via SUDO at all? Surely if we have (a) the user we are running as (ubuntu in this case) set to be part of the docker group, there should not be a need for SUDO, and therefore we won't have permission issues. Is this a valid argument? And (b) would it be possible to change "dst=data" to "dst=./data" so that the config and certs are created in a subfolder called "data" in our working directory? Thanks for all your beautifully crafted videos, they are an inspiration! 🙂
Thanks. It doesn't have to be run as sudo if you've added your user to the sudo group (that's up to you). Yes, you can use ./ notation if you want to create local directories, completely up to you how you want your folder structure.
So, A matrix isn't secure, they can tell who's in what rooms and when they're posting B they can't close their rooms because of how that process works to CP is just rampant
Matrix is definitely secure if you configure it correctly, remember that privacy and security are very different things. You can also restrict access if you want but by default it's designed to be open and federated (AFAIK).
Hello Jim, Firstly, thanks for your hard work. Lots of your videos help me to understand how to deploy easily dokcer container. I was able to deploy SYNAPSE without so much problem (except securityHeader middle middleware in Traefik that was "too" secure lol). But ... ! Everything is working in Synapse when 1 client is on the lan network. Since both client are outside of the line, call are ringing but you cannot hear anybody. Do you have the same behavior ? Looks like a TURN server is required, do you agree ? Or Am I missing something ? Regards,
Very useful thanks. Just one question - what was the database name specified in the config file - PSYCOPG2 - couldn't relate that to anything as I think you called the PGDB Synapse? For future content - really suggest the topic of enabling external calling as in my experience, that is always a nightmare. I assume it involves deploying coturn and that seems a black art.
Thanks. Yes, I haven't touched on coturn yet as my understanding is that NAT breaks it, which pretty much rules out home use. I do recommend jitsi though if you need a voice conference tool, I have a video on that.
Hi Jim! great video as usual. Thank your for making this one. I followed your instructions closely and I'm getting an error where when I'm spinning up the container, it says it can't find the homeserver.yml file (Config file '/data/homeserver.yaml' does not exist. You should either create a new config file by running with the `generate` argument (and then edit the resulting file before restarting) or specify the path to an existing config file with the SYNAPSE_CONFIG_PATH variable.) Wondering if you have any thoughts on this?
Thanks 👍 double check your bind mount for the homeserver.yaml file. This is something you can copy from my GitHub and tweak. It needs to be in the right place before it'll run.
hi. i am wondering how to increase the max file upload. i managed to increase it to 5gb but the upload doesn't start but fails after a little while. i am using the same docker image as you. could you help me?
don't use biometrics for access to your mobile devices especially. you are defeating (some of) the purpose of using e2e encryption. if one of you are using biometrics your conversation is vulnerable to your friend having a late night out.. passing out and someone unlocking the phone. worse.. if an officer asks you to unlock your phone.. and you refuse they'll 'accidentally' point it at your face.. so physical privacy with bio metrics is horrible.
From a purist perspective you're right, but most will lean on the other side of the security productivity scale. Besides, there are probably back doors in the OS anyway 😂
@@Jims-Garagethere are, but you don't want your girl friend grabbing your phone while you are sleeping, pointing the phone at you to unlock it and then sees your conversations with your wife.... i mean you may deserve it... or maybe enjoy it depending on how they take the news but no reason to make it easier. worse, someone knocks you out on the street to mug you find your wallet empty... put your thumb on your phone then drain your paypal account (or whatever). the average police department won't have access to those back doors... they can just ask amazon, amazon will share your data with anyone who asks haha 🙂 three letter agencies have access, especially if you don't fall into their political circle This is why you should just buy a phone from main land china. At least you know that they're stealing your data and probably not sharing it with the US government. Yeah what ever works for the user.. but the important part is that they know that any bio-metrics are a bad idea then decide what they want to do...
Choose whatever suits, but an external database is better (more performant) for larger servers. Sqlite becomes very slow as it increases in size (it's usually used for testing purposes).
sir i wanted to know how you will host each docker services like will you spin up new vm /lxc container for each service or you will host in a single vm docker instance
I originally had a Docker VM with all my containers. This is good for security (doesn't use the host's kernel), and portability. I now Kubernetes, with an 8 VM cluster split across 2 physical Proxmox nodes. I also have a HA Sophos XG firewall so if a physical node fails, my services remain up. I'll come onto this setup later in my video series.
The "proper" way is to use an .env file that you reference in the compose file. See here: docs.docker.com/compose/use-secrets/#:~:text=Getting%20a%20secret%20into%20a,on%20a%20per%2Dservice%20basis.
@@Jims-Garage ah sad! Maybe a tip: There's a bot called Draupnir that can be set up on a server to be able to moderate it better. Set this up for mine, too. Very handy! But of course involves still some time for moderation even though it is much quicker than per hand
24:40 - it looks like you might have a couple of mistakes in the homeserver.yaml user: synapse-db -- should be synapse_user ? the line cp_max:1log_con.... -- seems to be missing a line break after cp_max: 10, and seems to have the zero lopped off. Much docker logs -f ing ensued.
@@Jims-Garage thank you for the quick reply, much faster than many youtubers who don't respond at all! Can I still make sure they are the only ones with accounts and no on else can accses it?
I don't know a great deal about Jami, other than it shares similar features and mission statement. Adoption of matrix synapse is significantly larger, hence why I chose it. Adoption of new platforms (matrix is 9 years old) is always challenging, and larger populations often triumph.
Great video once more, really admirable how you are able to bring this complex material in an easy to understand way 👏🏻
Looking forward to your kubernetes set-up, as I think it would be great if I could just add more resources to these type of containers when needed. Indeed, bridging would also be nice, because it is hard to have others switch to matrix. Did you also considers dendrite instead of synapse? Mastodon would also be very interesting. Your channel is a true gem 💎!
Thanks so much, Robert. I chose Matrix simply because it is the largest, and most mature from the research I did, and my own experience (even though there are others that have similar features). I find if you're trying to convince people to jump ship it's easier when it's something that is "widely" used.
I will likely do a follow up on a Discord bridge as I'm going to set that up for my own Jim's Garage Discord server anyway.
Kubernetes is coming, there's just a few more 'core' apps and items I want to cover first. I know kubernetes is going to be niche, so I'm keen to have people fully set up on docker first.
echo the request for bridges - I also use telegram with bots for my internal alerting so bridging would be great.
I'd also agree a different piece of content would be a Mastodon server and maybe how to integrate with Matirx (I'm assuming they can as they both seem federated but maybe I'm just deluded 🙂
A major problem isn't technical - it's about getting your contacts to use alternatives other than the usual suspects. That perhaps is more of a challenge than actually installing and using the software.
I agree with you, adoption and mind switching is the hardest part. Hopefully I've called out how seamless it should be for people, but it's still a bit more effort than just downloading something 'that works'.
Hopefully the benefits are in line with the effort.
Absolutely, it's the same with Signal, I only managed to convice 10 people to use it in the last 2 years ...
Using matrix self hosted server for our business since the beginning of the project, highly recommended
That's great to hear. I wish it was more popular!
I also could not get in touch through matrix, but thank you.
This video helped me setup matrix synapse and I managed to figure out the proxy and whatsapp bridge myself.
Now I can finally read messages from WhatsApp groups on my iPhone without having to install that Meta stuff which i refuse to use. Sadly most people don’t care about privacy.
Thanks. Unfortunately I had to close down the matrix server as it was being spammed.
Thanks for the demo and info, have a great day
Thanks, Chris. Enjoy the weekend.
Good and clear explanation. I've been running Synapse for probably a year or so now as "one chat to rule them all"--I have the bridges installed for iMessage, Signal, Telegram, and WhatsApp, so I can use one app to communicate with any of those. Including iMessage on a Windows machine--that bridge is kind of fiddly to set up (and it looks like they're in the process of a complete overhaul of it), but it still works pretty well.
That's awesome, good job. How do you find the experience? Do any non-Techie friends and family use it?
@@Jims-Garage That wasn't really my purpose in installing it, so I haven't pursued having my non-techie friends/family use my Synapse server. Most of them are using my mail server, which is a start, and a few are using my self-hosted Bitwarden server. Once thing at a time...
I have it installed on an Oracle VPS, which is free since it's small enough. There's an Ansible installation that takes care of most of the details (except, at the time, for the iMessage bridge)--Synapse itself, Synapse Admin, web client(s) of your choice, bridges of choice, etc. Pretty comprehensive, but it's a lot of moving pieces.
Wow do tell
Awesome Thank you for Sharing! 💯✴
It would be great to see an update that includes using Redis to improve the response time of Matrix Synapse!
Thank you and God bless.
Looks good! Is there a follow-up planned with Matrix bridges? (looks like you don't really talk about them, seeing the chapters in the description.)
Thanks. Happy to visit some of them if there's sufficient interest.
Hey Jim. Love your videos.
Caption idea for your Merch: {it’s pretty straightforward}
Looking forward to a free hoodie for the idea 😅
Haha, thanks for the idea 😜
Nice solid video for getting yourself set up with a basic Matrix instance. Well done!
I've been running a server for a group of friends for just over two years now, it's been absolutely rock steady and has been surprisingly simple to keep maintained. Slowly over time I'm bringing more friends on-board.
It runs really smooth on an on-prem server with UPS backup and redundant failover networking, so has about 99.8% uptime so far (downtime is me tinkering with it lol).
Tried to drop you a message via the room link, but seems it's not there anymore.
Hey, thanks! It is there just I've had to put an aggressive geo block on it due to spam. I love it , just wish more people used it...
Excellent video thank you defo getting this setup
No problem 👍 Drop a hello in my public Matrix channel when you have it up and running.
@@Jims-Garage will do thanks
Great content, keep doing that
Thanks 👍
Hi Jim. Thanks, cracking video - I'm busy working through it to get Matrix installed. I have a couple of questions regarding the initial docker command at timecode 11:50 and onwards: Why does it have to be run via SUDO at all? Surely if we have (a) the user we are running as (ubuntu in this case) set to be part of the docker group, there should not be a need for SUDO, and therefore we won't have permission issues. Is this a valid argument? And (b) would it be possible to change "dst=data" to "dst=./data" so that the config and certs are created in a subfolder called "data" in our working directory? Thanks for all your beautifully crafted videos, they are an inspiration! 🙂
Thanks. It doesn't have to be run as sudo if you've added your user to the sudo group (that's up to you). Yes, you can use ./ notation if you want to create local directories, completely up to you how you want your folder structure.
Firstly thanks to Jim. On trying to build a new VM, receiving the following error: E: Unable to locate package matrix-synapse-py3
@@sturdyblock try to install it with apt
Hi Jim & thanks for replying. I've tried using: sudo apt install matrix-synapse-py3 ; but I receive the unable to locate error. Regards.
@@sturdyblock likely it's a pip3 package
You should do this on ubuntu and show everything you have, like if you are setting up a reverse proxy, full in depth tutorail
So, A matrix isn't secure, they can tell who's in what rooms and when they're posting
B they can't close their rooms because of how that process works to CP is just rampant
Matrix is definitely secure if you configure it correctly, remember that privacy and security are very different things. You can also restrict access if you want but by default it's designed to be open and federated (AFAIK).
Hello Jim,
Firstly, thanks for your hard work. Lots of your videos help me to understand how to deploy easily dokcer container.
I was able to deploy SYNAPSE without so much problem (except securityHeader middle middleware in Traefik that was "too" secure lol). But ... !
Everything is working in Synapse when 1 client is on the lan network. Since both client are outside of the line, call are ringing but you cannot hear anybody. Do you have the same behavior ?
Looks like a TURN server is required, do you agree ? Or Am I missing something ?
Regards,
Yes, calls require a turn server. You can either do that or check out my jitsi video (far simpler)
Very useful thanks. Just one question - what was the database name specified in the config file - PSYCOPG2 - couldn't relate that to anything as I think you called the PGDB Synapse?
For future content - really suggest the topic of enabling external calling as in my experience, that is always a nightmare. I assume it involves deploying coturn and that seems a black art.
Thanks. Yes, I haven't touched on coturn yet as my understanding is that NAT breaks it, which pretty much rules out home use. I do recommend jitsi though if you need a voice conference tool, I have a video on that.
Awesome video
Hi Jim! great video as usual. Thank your for making this one. I followed your instructions closely and I'm getting an error where when I'm spinning up the container, it says it can't find the homeserver.yml file (Config file '/data/homeserver.yaml' does not exist. You should either create a new config file by running with the `generate` argument (and then edit the resulting file before restarting) or specify the path to an existing config file with the SYNAPSE_CONFIG_PATH variable.) Wondering if you have any thoughts on this?
Thanks 👍 double check your bind mount for the homeserver.yaml file. This is something you can copy from my GitHub and tweak. It needs to be in the right place before it'll run.
I rechecked my binding and tweaked it and now everything is golden! Thanks 🙂@@Jims-Garage
I see
I got one now, how about manage user and their privileges guy, as default, new registered user can create room...
hi. i am wondering how to increase the max file upload. i managed to increase it to 5gb but the upload doesn't start but fails after a little while. i am using the same docker image as you. could you help me?
Honestly I run a matrix server and I still don't recommend it it's a complicated mess and channel load times are terrible
It's definitlely not for the feint hearted I agree!
I use signal is very good❤
don't use biometrics for access to your mobile devices especially. you are defeating (some of) the purpose of using e2e encryption. if one of you are using biometrics your conversation is vulnerable to your friend having a late night out.. passing out and someone unlocking the phone. worse.. if an officer asks you to unlock your phone.. and you refuse they'll 'accidentally' point it at your face.. so physical privacy with bio metrics is horrible.
From a purist perspective you're right, but most will lean on the other side of the security productivity scale. Besides, there are probably back doors in the OS anyway 😂
@@Jims-Garagethere are, but you don't want your girl friend grabbing your phone while you are sleeping, pointing the phone at you to unlock it and then sees your conversations with your wife.... i mean you may deserve it... or maybe enjoy it depending on how they take the news but no reason to make it easier. worse, someone knocks you out on the street to mug you find your wallet empty... put your thumb on your phone then drain your paypal account (or whatever).
the average police department won't have access to those back doors... they can just ask amazon, amazon will share your data with anyone who asks haha 🙂
three letter agencies have access, especially if you don't fall into their political circle
This is why you should just buy a phone from main land china. At least you know that they're stealing your data and probably not sharing it with the US government.
Yeah what ever works for the user.. but the important part is that they know that any bio-metrics are a bad idea then decide what they want to do...
You don't happen to plan to do a video on how to set up a turn server for getting voip/video working on this do you?
It's on the list but a little way down. There are instructions on the GitHub if you're looking to do it sooner.
Why you use a docker image for a separate db when synapse by default uses sqlite?
Choose whatever suits, but an external database is better (more performant) for larger servers. Sqlite becomes very slow as it increases in size (it's usually used for testing purposes).
Oh it would a killer feature if it accepted authentication with google accounts.
It does via OpenID: github.com/matrix-org/synapse/blob/develop/docs/openid.md
sir i wanted to know how you will host each docker services like will you spin up new vm /lxc container for each service or you will host in a single vm docker instance
I originally had a Docker VM with all my containers. This is good for security (doesn't use the host's kernel), and portability.
I now Kubernetes, with an 8 VM cluster split across 2 physical Proxmox nodes. I also have a HA Sophos XG firewall so if a physical node fails, my services remain up. I'll come onto this setup later in my video series.
How do manage user and passwords in .yaml files? Do you leave them in there or do you just remove them after you have ran it?
The "proper" way is to use an .env file that you reference in the compose file.
See here: docs.docker.com/compose/use-secrets/#:~:text=Getting%20a%20secret%20into%20a,on%20a%20per%2Dservice%20basis.
What about the plain text password in the home server.yaml file?
Did I miss something? I changed the password and all I get are password authorization problems, my user is unable to authenticate.
Synapse up and running including working federation. I couldn't join your room though and your server URL is getting errors on Federation Tester.
Great job getting it working. Sadly I had to close it due to abuse. Perhaps in the future :)
@@Jims-Garage ah sad! Maybe a tip: There's a bot called Draupnir that can be set up on a server to be able to moderate it better. Set this up for mine, too. Very handy! But of course involves still some time for moderation even though it is much quicker than per hand
24:40 - it looks like you might have a couple of mistakes in the homeserver.yaml
user: synapse-db -- should be synapse_user ?
the line cp_max:1log_con.... -- seems to be missing a line break after cp_max: 10, and seems to have the zero lopped off.
Much docker logs -f ing ensued.
Thanks. Always check the GitHub for latest versions and fixes.
@@Jims-GarageBTW - do you have a guide on using tokens for authentication, instead of using anything google?
Hi, great video, can I just let my friends and I use my home server or can anyone use it?
You can create private rooms etc and disable registration
@@Jims-Garage thank you for the quick reply, much faster than many youtubers who don't respond at all! Can I still make sure they are the only ones with accounts and no on else can accses it?
@@sosoh329 yes, check this out: matrix.org/blog/2019/11/09/avoiding-unwelcome-visitors-on-private-matrix-servers/
@@sosoh329 you can also put it behind a VPN
@@Jims-Garage Thank you!
is this more secure than briar ?
How does this compare to Jami?
I don't know a great deal about Jami, other than it shares similar features and mission statement.
Adoption of matrix synapse is significantly larger, hence why I chose it. Adoption of new platforms (matrix is 9 years old) is always challenging, and larger populations often triumph.
synapse has been archived, what now?
Really, can you link it?
seems that it's still up?
👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍!!!
👍