The Challenges of Writing a Massive and Complex Go Application
HTML-код
- Опубликовано: 17 окт 2024
- Author: Ben Darnell
Abstract:
We opted for Go when building CockroachDB, a scale-out, relational database, because of its support for libraries, interfaces, and tooling. However, it has come with its own frustrations, often related to performance and synchronization. And as for Cgo, RocksDB, and other critical external libraries, we've had to hunt down or develop creative workarounds to ensure they work well the rest of the toolchain. In this talk, we'll share how we've optimized our memory usage to mitigate issues related to garbage collection and improved our use of channels to avoid deadlocks. We will also share creative techniques to integrate non-Go dependencies into the Go build process.
ACM DL: dl.acm.org/cita...
DOI: dx.doi.org/10.1...
How is building application in go is fater than java? any insights would really help.
Curious what exactly the style was that helped alleviate bugs with having incorrectly assumed inner fields were mutable
There used to be a joke that you could make FORTRAN programmers write in another language, but they would still write FORTRAN programs, just with different syntax. This sounds a lot like making C++ programmers write in Go, and they wrote C++ programs using Go syntax. Personally I think C++ syntax is horrible enough to make that sound like not a terribly silly thing to do.
Excellent talk.. good stuff
I'm building a CNN using Go, and I was having the same issues that they were running into. Especially the memory issues with slices, and the deadlocking with goroutines.
@Andrew Odiit Convolutional Neural Network
Excellent talk !!!
Thanks for the level of detail, and congrats on reaching 1.0 and production readiness.
The Jepsen write-ups are at www.cockroachlabs.com/blog/cockroachdb-beta-passes-jepsen-testing/ and jepsen.io/analyses/cockroachdb-beta-20160829.
Dan's discomfort to use Golang to rewrite CockroachDB is scary.
All I can say is bunch of immature decisions with not enough technical reasoning
+kk8866 I'm not sure what you mean by that comment. Going back and thinking about SQL Server and OracleDB and the millions of decisions that they had to make... Oracle had to build their own RAW disk drivers in order to improve performance where today's flash and SSD might have leveled the playing field. Similarly most performant DBs typically allocated ALL of memory and built their own memory managers; many predating virtual memory systems... plenty of bad decisions there.
Since immature decisions are typically made by immature people.let's consider what it takes to work for any silicon valley type startup.... [a] overconfidence in one's abilities [b] "OK to fail" millennial ideals [c] willingness to work 100 hour weeks for marginal salaries and free meals. [d] maybe 10,000 hours as a programmer or architect but certainly not 30K+. These attributes to not describe journeyman programmers.
Personally I have been a full-time programmer for over 30 years and with the exception of my honeymoon I have worked no less than 40 hours a week and since it's my hobby I have always put in another 20-30 hours doing the exact same; much to the chagrin of my wife and family.
SO IF IT WERE UP to me.... I would have constructed some sort of DSL or MACRO/template system so that the "design" was isolated from the implementation. This potentially solves many problems but also creates a few. First of all it's easier to test. Secondly there is a CHANCE that the code generator could replaced with another and have the same outcomes. (LLVM is interesting proof of that). But you have more to do and more to get right.
Wish you and your family all the best!
@@mateuszkurowski45102:29
Time to rewrite it Rust.
and make things worse?
@@samdash3216 rust projects are a lot more structured/opinionated in structure. Also Rust is a more capable language imo. Not to say that Go has good applications.
Why can't some technical people give proper talks? All those ahms and repeated words really detract from the talk.
Almost unbearable to listen ...
Why can't people avoid making comments like this on RUclips videos? Why can't said commenters understand that the presenter(s) in question may have been assigned the presentation duty, and didn't choose to do it? Why don't people understand that not everyone is a great presenter, and that THAT'S PERFECTLY OK?!
Elon Musk is not good at public speaking but it doesn't matter as he has important things to say.
@@rsilvers129 Are you kidding? Or do you mean like wanting to go private at 420? Or accusing someone of being a pedo? I won't hold it against him, but I will hold it against you.
No, I didn't mean either of those things as examples of important things that he said. Those were bad things. I was talking about good things.