Free Dissociation
Free Dissociation
  • Видео 10
  • Просмотров 37 493
A Software Engineer Reacts To Star Citizen Server Meshing & Replication Layer - CitizenCon 2953
By popular demand (thanks, both of you ;) I share some thoughts from a software engineering perspective about the Replication Layer and Server Meshing section of the StarEngine talk from Star Citizen's CitizenCon 2953 this October 2023.
2023 CitizenCon presentation on replication layer & server meshing:
ruclips.net/video/xKWa4WoTkV4/видео.htmlfeature=shared&t=4500
2021 CitizenCon presentation on persistent entity streaming: ruclips.net/video/TSzUWl4r2rU/видео.html
Jake Beal paper on "What the Assassin’s Guild Taught Me About Distributed Computing" from MIT: jakebeal.github.io/Publications/MIT-CSAIL-TR-2006-038.pdf
Overwatch Developer Update | Let's Talk Netcode: ruclips.net/video/vTH2ZPgYujQ/в...
Просмотров: 32 030

Видео

Arrow takes down a Valk - Bounties - Star Citizen - 2022.12.31
Просмотров 1019 месяцев назад
Arrow takes down a Valk - Bounties - Star Citizen - 2022.12.31
Star Citizen 2022 12 30 23 58 00 04 DVR
Просмотров 509 месяцев назад
Star Citizen 2022 12 30 23 58 00 04 DVR
A Software Engineer Reacts To Star Citizen's Graph Database Woes
Просмотров 4,3 тыс.9 месяцев назад
In which I attempt to provide some background context from a software engineering perspective to the recent post from Chris Roberts about the struggles Star Citizen is currently having with the graph database underneath the Persistent Entity Streaming service. CitizenCon presentation on persistent entity streaming from 2021: ruclips.net/video/TSzUWl4r2rU/видео.html @BoredGamerUK video with Chri...
Nice bounty run - Star Citizen - 2021.12.03
Просмотров 529 месяцев назад
Nice bounty run - Star Citizen - 2021.12.03
We Added EXTRA PARTS, Why Is It Still FAILING?! - Redundancy is Not Reliability
Просмотров 2973 года назад
Why adding redundant parts doesn't always make systems less prone to catastrophic failure. Engineering a Safer World, by Prof. Nancy Leveson: mitpress.mit.edu/books/engineering-safer-world (open access PDF at link)
In Which I Install a WiFi Card (and Some Other Stuff)
Просмотров 283 года назад
Installing a WiFi card, a tower cooler, and a new RAM kit. Pretty much what it says on the tin. None of these are affiliate links, that sounds like work. Hardware I'm installing: Fenvi AX200NGW WiFi 6 & Bluetooth M.2 - www.newegg.com/fenvi-fv-ax200h-m-2/p/0XM-00JK-00063 G.Skill Ripjaws V DDR4 3600 64GB kit (4x16GB) - www.newegg.com/g-skill-64gb-288-pin-ddr4-sdram/p/N82E16820232927 be quiet! Dar...
San Francisco Giants Stadium Sunrise-to-Sunset Timelapse 4K
Просмотров 5073 года назад
Filmed in Mission Bay August 15, 2020 San Francisco, California Get this as a (free!) dynamic wallpaper for macOS: www.dynamicwallpaper.club/wallpaper/28hjb5f3i87 Music used, in order of appearance: "City Lights" - Ross Bugden ruclips.net/video/ryAuJ2aiWsw/видео.html Used under CC-BY Creative Commons Attribution 4.0 International License (creativecommons.org/licenses/by/4.0/). "Icy Vindur" - A ...
Ranting About Power Supply Reviews and Installing Some Cable Extensions
Просмотров 623 года назад
Tom's Hardware review I'm using as an example to critique - www.tomshardware.com/reviews/corsair-rm650x-psu,4611.html (noise chart is on page 5) (none of these are affiliate links, because I'm lazy, and good luck finding many of them right now :-/) Hardware I'm installing: be quiet! Straight Power 11 650W Platinum - www.bequiet.com/en/powersupply/1776 Asiahorse Sleeved Cable Extension Kit - www...
Space Animation
Просмотров 10513 лет назад
When I was a freshman in high school, circa 2000, and thought I wanted to be a computer animator, I spent a long time playing around with POV-Ray. This is the movie I made. It was rendered on a Pentium III 667MHz with 128MB of RAM running Windows 98SE.

Комментарии

  • @Ic3q4
    @Ic3q4 2 месяца назад

    do you know which specific one they use? bc i can't remember

  • @IntegralTriangle
    @IntegralTriangle 2 месяца назад

    bullets are physicalized, yes.

  • @x0myspace0x
    @x0myspace0x 2 месяца назад

    I'm not really surprised that it works in a test setup. You can get anything to work in a test setup. What I'm wondering is whether it will keep working when this gets upscaled to thousands of simultaneous players and millions of individual objects being streamed between different clusters and the interactions between all those entities. Time will tell...

  • @KnoWareMan03
    @KnoWareMan03 2 месяца назад

    This has to be AI + quantum combined. This is beyond human comprehension. Besides that, there is a curvature visible. IMO😵‍💫 *Ok...this is the Oasis🤔 Right?

  • @Crittek
    @Crittek 3 месяца назад

    The replication layer is also moving the client using its input. Most of their idea is ripped out of Unreal Engines networking, everything is server authoritative. As a consequence, what this does and nobody is talking about is enabling very effective anti-cheat measures.

  • @nickstinger4709
    @nickstinger4709 4 месяца назад

    Ugh. Everything old is new. The replication server is nothing more than a glorified IRC server with a messaging and hierarchicy and filters. You could also describe it as an event system similar to windows or Javascript events. I don't understand why it's taken them years to re-engineer something that's open source and stable already. Though I will say, this is the most fun part of the entire Star Citizen project. The real work will come from it's application. If it takes 5 years to build a chat server, the stuff attaching to it should take another 50.

  • @kemo_l7350
    @kemo_l7350 5 месяцев назад

    How could I get a job with cloud imperium games?

    • @kemo_l7350
      @kemo_l7350 5 месяцев назад

      I mean that I don't have that many skills and I am in Jordan so the only place I can can learn from is the internet

  • @kevin-jm3qb
    @kevin-jm3qb 5 месяцев назад

    can you please teach me computer science? :D

  • @sketchtherapy1218
    @sketchtherapy1218 5 месяцев назад

    Would you please do a video with Mr Tybio about the 5 toughest problems in computer science CIG is taking on or 1 video for each of the 5 with Mr Tybio?

  • @DevilbyMoonlight
    @DevilbyMoonlight 6 месяцев назад

    at the end of the day, your packets are at the mercy of the routing and traffic shaping policies of every autonomous system they traverse,

  • @qdpqbp
    @qdpqbp 6 месяцев назад

    my english not that good you said youy make perfect the enemy of the good? if so you just described my life

  • @marclamy364
    @marclamy364 7 месяцев назад

    Players, just like servers, have a streaming container around them. It is likely that servers only need to see around them to facilitate simulation and transitions, but players have variable needs. For instance, you'd expect a player flying at 1200m/s to need a much bigger container to stream entities early enough to avoid popping. So to answer your question, if all 3 servers in this demo were in a straight line (purple, green, red) and bullets were physicalized, you could imagine a player in server purple shooting at a player in server red. The server purple would manage the physicalized bullet for a while, then server green and then red, potentially handling the collision with another player there. While the shooting player needs to see the result of their shot from server purple to red, server purple doesn't care. So they'd have very different streaming containers.

  • @marclamy364
    @marclamy364 7 месяцев назад

    You're correct about the fact that running all servers and the replication layer on the same machine, it eliminates the latency issue. That said, the latency between game servers and the replication layer, assuming running on the same data center, should be negligible. Assuming the game servers simulations run at approximately 60 ticks per second, that gives around 16ms per tick. With an average network latency of, say, 1ms within the same data center, and standard client-server synchronization techniques applied (client prediction, server reconciliation), it is very possible that no latency would be visible in this very simple demo.

    • @free-dissociation
      @free-dissociation 6 месяцев назад

      Wellllllll. It doesn't _eliminate_ the latency issues, unfortunately-I wish it did. There's all kinds of nonsense that gets in the way. (Whatever nonsense the scheduler has decided it's getting up to that day is the one that I remember biting me personally.) But I'll certainly take same-machine latency over even same-datacenter latency.

  • @TheHorodateur
    @TheHorodateur 7 месяцев назад

    Audio is horrible

  • @Issodigga
    @Issodigga 7 месяцев назад

    To be honest. I think the way at the Moment ist Just a Server maybe for a Planet and surroundings moons. So for the Moment you would have 4 Servers vor crusader and 4 for Pyro. And the Transition ist easy hid in the quantum jumps. So 100 Players in every Planet System is quit good for now. And if you need you seperaten a moon for more Players there. And If a Planet is empty from Players you Stream the hole Thing Out and shut the Server down. Throu the quantum jumps the streaming ist very good Hidden. The Same way you could so spacebattles in a bubble.(Server) So i dont think they will start streaming every room an Split two rooms in two Servers when there are Just 100 people. Have a good day Guys.

  • @Shawnsrumi
    @Shawnsrumi 7 месяцев назад

    As SC backing I appreciate your perspective…thanks for the insight

  • @HitmannDDD
    @HitmannDDD 7 месяцев назад

    We saw a demonstration that carved up the game space by 3d spacial boundaries because it's easier to visualize in a demo. However, given what you know about the tech, what additional technical challenges do see regarding removal of the spacial boundries for servers? The use case being that multiple servers could then handle a single 3d gamespace, thus potentially increasing the max player count in close proximity.

  • @harrywyependragon4242
    @harrywyependragon4242 7 месяцев назад

    I don't trust anything they showing in this shows for all those years since 2016 by the reasons. Better try live game and see how it's really looks and works by your own eyes cuz it's socks and that's not only my opinion. And I don't trust in their promises anymore as well. 🧦

  • @MrJabbaStyle
    @MrJabbaStyle 7 месяцев назад

    this is the stuff that i don't understand at all but still understand enough to be highly impressed!

  • @consumablecorner150
    @consumablecorner150 7 месяцев назад

    Firs time watcher. "Perfect is the enemy of the good." Never heard that before! "..the truth will set you free." Definitely an empowering statement. You're a very soft-spoken bro. In fact, so much so that I could hardly hear you xD (edit: at least in the beginning)

  • @Tarkovclips-mw7ur
    @Tarkovclips-mw7ur 7 месяцев назад

    no sound

  • @Deva-Jufan
    @Deva-Jufan 7 месяцев назад

    Stop saying sorry for things in the background. It's hardly noticeable.

  • @craigwoodward8455
    @craigwoodward8455 7 месяцев назад

    7:16 CIG dont have a habit of avoiding bugs during live demos... where'd you get that? It's on the bingo for one of their live demos to crash. XD It almost always messes up for them, the amount of times its crashed entirely during the demo has been a meme. lol

  • @RayTieRom
    @RayTieRom 7 месяцев назад

    Excuse my ignorance, wouldn't server meshing solve this issue? Essentially dividing the workload and database requirements for each server, no?

  • @JohnMcclaned
    @JohnMcclaned 7 месяцев назад

    memgraph is the only thing i can think of that tries to do graph in real-time

  • @sc_cintara
    @sc_cintara 7 месяцев назад

    The community really needs a youtuber to go through the SQ42/PU monthly reports engineering section...

  • @DamonCzanik
    @DamonCzanik 7 месяцев назад

    Asimov said any sufficiently advanced technology is indistinguishable from magic. And even understanding the fundamentals I'm still in awe. It's magic. Thank you for this video. While yes, it could have had better production quality and editing, that's not what I was looking for. I learned a lot and enjoyed the wonderfully nerdy discussion. I had so many questions that were answered and learned a lot. This was the next best thing to having the actual developers. It took me a few times to get through this, not because I didn't understand but sometimes you need to devote your full attention to something. And that can be difficult for people with busy schedules. This is not the kind of video you have going on in the background while you work or even play games. That being said, what a treat. Just geeking out and learning something. I know traditional relational database management systems but graph databases are things I never needed. So the advantages and disadvantages were foreign to me. I knew it was fast, but why? What did it have over a MySQL database? It's highly interconnected structure is particularly advantageous for applications requiring real-time scalability and complex relationship management, like server meshing in games. I have done back end server programming on applications and game programming in Unreal, Unity, etc. But never back end server programming for multiplayer games part. So this was all new, and exciting to learn. The best analogy I can come up with, is databases like MySQL, SQL Server, etc. are laid out more like Excel files, while graph databases are laid out more like Russian nesting dolls. Like the dolls that seamlessly nest within each other, graph databases store data in a way where entities (nodes) and their relationships (edges) are directly linked and nested within the network. This structure seemingly allows for easy navigation through deeply interconnected data, similar to opening one doll and finding another connected one inside. It's probably a lot faster to move data around than an SQL join. It also makes me appreciate the complexity of the task even more. I loved learning the likely reason for the L shaped level. I hadn't even given it a thought. But it suddenly makes so much sense. I had a "Oh. Wait, Oh! Oh!!! Now I get it!" moment. Yeah the seamlessness would be broken if you could see them all in a straight line! The user in purple zone would see things just disappear if they were in the green zone transitioning to the red zone. And when looking at it from the purple zone, things transitioning from the red to the green zone would seem to suddenly appear. While the first, second, and even third iteration of server meshing won't likely suffer from much of this. It could be an interesting problem for dynamic server meshing. Knowing the limitations may reduce the magic-ness of it all, but it's still no less amazing. I know about live demos. I had to give a live presentation to executives at Chevron (yes, that Chevron) when my application started dying on me. It takes a lot of guts to demo it to 3 thousand people, and 30 thousand+ watching online. Thanks for the video. I truly appreciated it. Even the other video that had the sound issues. Lol. It happens to all of us man. Nothing like the big "technology guy" has trouble with technology. The problem with going to MIT is that people can always use it against you. "Mr. MIT can't even find the ketchup when it's right in front of him!" And of course, people assume you're just brilliant at everything. They call me magic man at work. I enjoy the compliment, but I am not. I still feel intimidated about walking up to some massive complex network printer I've never used before. So despite all the technical issues, it was worth it for me. Because I really enjoyed these videos.

  • @bjordan429
    @bjordan429 7 месяцев назад

    Good video. Was looking for a software opinion on this. And double thanks for highlighting which part was 'rigged'. 2 side notes: - 'note card' your intro and talking points. You seem fine once you get past that and into your niche - Tuning your audio can help with your audio environment. I've recently started using OBS as well but found great results from the Noise suppression options in...less-than-private environments. ruclips.net/video/3rcAPp_8WkU/видео.html

  • @TheKingoftheriff
    @TheKingoftheriff 7 месяцев назад

    "lightning bolt, lightning bolt, lightning bolt" Fucking subscribed.

  • @surslaughter3524
    @surslaughter3524 7 месяцев назад

    Great video. I have one issue though. Saying CR didn’t risk anything when speaking about anything in SC is wrong. He put everything on the line. From most of the gaming community screaming scam for 10 years. Half of the SC community doing the same. Every issue, every marketing disaster. Every failed patch. Every projected release date for ships/features/patches. He put his name and career on the line every day for 10 years. He deserves some credit.

  • @IronPhysik
    @IronPhysik 7 месяцев назад

    I have a feeling that you may have misunderstood the job of the replication layer in all of this the way I myself understand it is that the RL is there to replicate objects for other servers on other zones so that each server can do the physics calculations for its own zone of objects, and it just displays the results of what the others did so its "replicates" objects for the player, and thats how you can shoot across boundries.

  • @doubledigital_
    @doubledigital_ 7 месяцев назад

    player can see it because the replication layer is now showing all clients all the logic ,, but the server its self has streamed out the data.. saving a fuck load of memory the replication layer

  • @doubledigital_
    @doubledigital_ 7 месяцев назад

    the server is running on the laptop that is true but the fact they even have it working will unstress the servers .. we can have as many servers as we want now.. not just one getting hammered lol nu us lot.. you can even apply this to space battles if needed so when a fuck load of people are in a small space you just spool up enough servers to deal with the traffic bobs ya uncle ^_^

  • @doubledigital_
    @doubledigital_ 7 месяцев назад

    been faking levels for ages because of memory only recently are we able to do open worlds that we dont need to cheat :D fps games like quake we use to hide parts of the level and you would only see it when you moved into the box show here.. the interesitn part is they have linked the box ( container ) to the server its self.. i aint ever seen it done this way before and i been doing game shit for 25 years give or take. impressive shit.. this in fact impressed me more than anything.

  • @Yokeus
    @Yokeus 7 месяцев назад

    I would be curious with how big CIG decides to make these zones. If the red zone is 1 Million meters across for a whole planet, you wouldn't be able to shoot across three zones like you had mentioned. The weapons and scanning don't have those kind of ranges at the moment.

    • @doubledigital_
      @doubledigital_ 7 месяцев назад

      the solar system its self is a giant zone, they can nestle everything .. they just need to link them all to servers now.. fiddly but its more or less done :))

  • @JohnZingTTV
    @JohnZingTTV 7 месяцев назад

    this video convinced me to install the free 14 day epo install and try this game even with my poopy 4gb 1050 gtx ti gfx card keep up the content i think the 20k views show there is a high demand for this content and the format it was very real i look forward to see whatelse you will cover as far as other games are concerned. well next convention if yougain alot of traction on youtube mabey they will pay all exspenses so you can go taere nextime to citcon right? :D

  • @a__random__person
    @a__random__person 7 месяцев назад

    in future, since you're recording a youtube video in 4k.. you should either upload in 4k as well so we can read the text in the video easier.. otherwise reduce the source video to 1080p :)

  • @memorycl
    @memorycl 8 месяцев назад

    Fairly sure you're correct about neo4j...simply based on the number of version updates that have been pushed in the past 2 years vs previous years :) en.wikipedia.org/wiki/Neo4j#Release_history

  • @josephbaker6461
    @josephbaker6461 8 месяцев назад

    The bullets in Star Citizen are indeed individually modeled, probably for the reasons he said. If you play with the active camera in game you can see it.

  • @Turican76
    @Turican76 8 месяцев назад

    server meshing is 20 years old tech in EVERY mmo nothing new

    • @Sparkk0
      @Sparkk0 7 месяцев назад

      K. Would like to see your 20 year old example of shooting something real time from one server to another with another player in the background hopping around.

  • @artuno1207
    @artuno1207 8 месяцев назад

    "This is not something a AAA publisher would have put up with" this is why Ive believed in this project. This why ive been patient. Theyre doing soemthing phenomenal. I want them to succeed so that all the haters of this project and those who call it a scam have an amazing game to play soon(TM).

    • @sc_cintara
      @sc_cintara 7 месяцев назад

      Yes, the funding model is the real secret sauce of Star Citizen. No financer would approve of waiting this long to start selling the game if it had been done in the traditional way. All that money sitting in CIG would be money not giving of interest, so the expected profit requirement for the game would grow year on year until they become untenable. At that point the game would have been forced to publish to recoup as much of the loss as possible. It is the main reason why Forbes hates Star Citizen; the funding model entirely cuts out the financer.

    • @r2artCH
      @r2artCH 6 месяцев назад

      I think the game already good with all the current features. Only one system but there's so many things to do. There is only two things that bother me with the current state, and that is the bugs and the NPC AI. If they fix those two, I can say the game already complete even with only one system. With this player already able to play the game, while they working on expanding to a new system and new vehicle internally until it's ready, stable and minimal bugs they can update it then continue expanding internally until ready. Just keep doing it like that, it should be good.

    • @sc_cintara
      @sc_cintara 6 месяцев назад

      @@r2artCH Unfortunately, having each release mostly bug-free will slow down development quite dramatically. Think of it as building a something like a chest of drawers. You build it with some group of drawers to begin with, polish it, make sure all the nails are tightly nailed in, etc, and give it to the customer. Then you want to add new drawers. To do that you have to break apart much of what you already built to fit the new drawers. Then you have to re-polish it, fix it up, etc to make it look nice again. This is the problem with releasing code under development: If you want it to be bug free and polished, you have to redo the bug-fixing and polish every time you release and that slows down development dramatically. So CIG are doing minimal polish and accepting quite a lot of bugs, especially in systems that are planned to be replaced, to avoid getting all resources stuck in bug-fixing and polish.

  • @PlumHeadLJ
    @PlumHeadLJ 8 месяцев назад

    51:20. I think there is no bullets, just raycasting outputs from player entity with saving the angle (vector) when leaving one zone and receiving it by another zone

    • @TheKingoftheriff
      @TheKingoftheriff 7 месяцев назад

      You also need a timestamp and an owner, bc the player isn't on the second server.

  • @zychrias
    @zychrias 8 месяцев назад

    You're not thinking in scale and are getting hung up on the "dog leg" of the purple versus red. This demo is small and concise to get the point across of Server Meshing operating. Servers are going to be controlling planetary bodies, stations, etc. Not just one tiny section of one hallway. A weapon fire from purple, thru green, hitting something in red (if your preferred arrangement of red being inline instead of dog legged around) is going to do exactly the same thing as firing from purple into green (like shooting the buggy in this demo showed).

  • @carllong8954
    @carllong8954 8 месяцев назад

    "distributed system is the hard part" That's why we have all been waiting for this game for 12 years now. I can remember way back at the beginning when CR was talking about having 2 cap ships fighting and each was a server and shooting from one ship to the other would involved hand offs between 3 different servers at the shot moves. That is why this demo might be the biggest step in the development of this game. This demo showed everyone who understood and has been waiting that it may actually be possible.

  • @mmmaxxx__
    @mmmaxxx__ 8 месяцев назад

    Is this technology something that is ground breaking even for worldwide day-to-day life operations like product inventories for big retailers, warehouses, logistics, finance, and definitely other small to massive games that might benefit from server meshing? What CIG have accomplished, would it have a moderately big or massive impact in some industries?

    • @free-dissociation
      @free-dissociation 7 месяцев назад

      My read is that it's pretty limited to virtual worlds-ops stuff like you're discussing is already pretty well handled by existing database tech, and this doesn't improve on that. But, if this works, then many other virtual worlds at the scale and complexity of Star Citizen may follow.

  • @Dovahkiin53
    @Dovahkiin53 8 месяцев назад

    Thanks, well explained and thoughtful.

  • @Red-Wyrm
    @Red-Wyrm 8 месяцев назад

    Very hard disagree about live builds. Gamers are not morons, and they easily distinguish vertical slice showcase. And with that - trust won't be at as desirable level as developers might want from gamers. On the other hand, show them a buggy mess, but honest showcase of live build - and gamers tend to be quite forgiving on most bugs they will see. And with that - retain a trust from gamers. Because when you show live build - you treat gamers as they are on the same level as you, rather than "Oh, those morons won't get it anyway. So, let's just show them only pretty parts". As a example - watch last ExileCon. Where devs played live build (of upcoming Path of Exile 2) on stage. And even later invited other random player to play it on stage as well (who is actually managed to play better than dev XD). THAT is level of trust that makes Path of Exile really stands out. Not the only game, and not the only gamedev studio like that. But just as example.

  • @sebhirsch4010
    @sebhirsch4010 8 месяцев назад

    Stereo!

  • @gorganhorn6872
    @gorganhorn6872 8 месяцев назад

    Please tell me your uncle is Jason Issacs, because that would make perfect sense in this reality. BTW thanks for your expertise and taking the time to unpack all the details that even CIG's show cannot do. +1 sub

  • @mr.potato9449
    @mr.potato9449 8 месяцев назад

    As someone who doesn't know much about programming, what does this mean in regards to server player limits? Say the server limit is 100 and call this demo a city. Without meshing you could only have 100 people in this city so would only see and interact with 99 people. From what I gather of it, with the city now being 3 zones and a different server having authority over each zone then you could have 90 people in the purple zone and 60 in the green zone so if you were in the purple you would now be able to see and interact with 149 people and not just the 99 you could when it was just 1 server with no meshing. Is that correct or am I way off? If that is correct then the servers still have the 100 limit so what happens if there are 100 people in the green zone and you move from the purple zone to the green zone?

    • @free-dissociation
      @free-dissociation 7 месяцев назад

      You're correct! The goal here is to be able to substantially increase the player count on what we've called "servers" up to this point-what CIG are now calling it a "shard", and it is made up of multiple servers. Ideally in the case you describe you can see and interact with all 200 people (100 in the green zone and 100 in the purple zone, each on their own server) seamlessly as though they were all on a single server.

    • @mr.potato9449
      @mr.potato9449 7 месяцев назад

      @@free-dissociation So what will happen when you move into a zone which is already full? Is there some kind of dynamic system where it will split that full zone into smaller zones (which now have their own server and limit) until there server in control of the area your entering is no longer full? If something like that is the case I would think there would have to be some kind of noticeable latency with the amount of times they might have to move people from server to server if an area is full and after the split other people have already entered it making it full again and needing additional splitting and so on o.O