I hate that PeaZip does not support passwords with a quotation mark (") in the UI. I haven't reviewed the code, but I am assuming that they are neither passing the password via a variable (linked library) nor via an execve call (less secure), but via some sub-shell which is a common attack vector and bad security practice. But as I said, I haven't reviewed the code.
I haven't ran nor looked at the code for PeaZip, but the website describes the program as a GUI wrapper around command line utilities. I highly suspect it's the sub-shell thing.
I guess you can call it that. You can start it on all your devices (e.g. laptop, PC, phone, tablet, Docker containers, etc.) and it will assign a fixed IP to each of them. Using this fixed IP all those devices are able to communicate securely with each other, no matter where those devices are located at the moment. Tailscale discovers the fastest, direct route between each device and routes the packages accordingly.
Example: Start it on all your devices and you can always access the shared files from your PC at home on your phone or tablet, no matter where you are. If you happen to be at home, no packages will be sent over your internet connection (so you can easily use it to stream 4k movies for example), but the way to access those files is always the same (fixed IP) and always secure.
An additional advantage of Wireguard (which is used behind the scenes) is that it maintains a strict mapping of IP addresses and identities. Therefore, you can usually restrict access to services by IP (instead of using TLS certificates or complicated authentication mechanisms).
Thanks for the blog post. I am following the project closely since Brad announced working for you.
My main concern currently is the coordination server which does not fit the zero trust claim.
I know that the traffic between peers is end-to-end encrypted and you did a good job designing your DERP protocol. However, the ability of the coordination server (login.tailscale.com) to add arbitrary nodes to my private network without my consent scares me.
Maybe you can use Wireguard's PSK to add an additional pre-shared-key to all nodes that is not managed by the coordinator (and never transferred to it)? This would make the setup slightly more difficult (you need to login to tailscale.com AND you need to provide your PSK), but it would at least ensure that the system can never talk to foreign nodes added by the coordinator itself (because the foreign nodes do not know the PSK).
If you already use the PSK for something else, another passphrase/key-file that is never transmitted and is XOR'ed over the PSK will do the trick. The firewall configuration that is pushed to all clients should be probably signed by a local key too, since it is rather critical.
Another possible attack of the coordinator would be if he pushes a configuration with correct VPN IPs and correct public keys, but changes the mapping between them. Since the IP within a wireguard network is usually used as an identity, this might be a huge problem.
Well, this just listed out my concerns pretty well and I spent the last hour installing this on my devices. Specifically, how do I explain to a fortune 500 IT department how this is secure in a few simple sentences?
Large companies that want to remove us from the trusted group should run the coordination server on-prem.
We have considered other certification options, but so far they boil down to running a part of the coordination server on-prem. Still exploring the space though.
Sure, large companies can easily run a on-premise coordination server. But large companies usually also have a network department and several other SDN and VPN solutions already. They are probably using highly tuned and standardized ipsec tunnels with hw accelerated AES encryption and special devices from different vendors.
I think tailscale could become the #1 solution for personal users and small / mid-sized companies (some of them might not even have an IT dept). At least the easy installation, the single process design (no separate IKE service etc.) and the opinionated modern cryptography - that deliberately does not allow any configuration at all - would make it a really good candidate for those use cases.
This users can probably rent a cheap $3 VPS easily and use it as an on-prem coordination server. However, this increases the installation effort considerable and trusting the cheap $3 VPS to be an essential part of your cooperate network might be a deal breaker. Many of those users might not even have a dedicated team for server maintenance and might not apply security updates regularly.
So, please add an (optional) additional PSK (or key pair / certificate) to each node that is not shared with the coordination server and can be used to sign / verify the configuration on each client. Users that do not care don't have to specify anything (they just have to log-in and everything works). Users that do care would have to login and provide the PSK on each device.
(And to be clear, the problem is not that I do not trust you. I am sure you are doing a great job and I would really like to use your service. But you are an US based company and with the new data protection laws within the EU it might not be easy to convince all customers that potentially sending all their data to a third-party company within the US is necessary.)
The most obvious pattern, a simple for loop and a separate iterator type, is imho missing. Something like "for iter.Next() { fmt.Println(iter.Value()) }" is commonly used in Go and doesn't look that bad either...
I'll buy that -- it's sort of mentioned in the post as an object-based wrapper around the closure version, except I listed the functions as HasNext() and Next(). More boilerplate for the iterator implementer, but I agree, it reads better for the caller.
Go's GC isn't nearly as powerful as any of the many GC implementations for the JVM, but Go's approach is also quite differently. In Go, you can reduce / avoid the garbage that you create. And when the GC turns out to be the limiting factor of your cache, then you can also mmap some dedicated pages from the OS and manage the memory on your own. But I guess that won't be necessary. Do you have any issues with Go's GC today?
Well I don't have issues with Go's GC today because my horrific experiences with various JVM's has been a powerful deterrent for any future experiences in performance critical large-heap server processes with a GC controlled heap. Other than that, I really really like the design of Go. This is in addition to the factor of my faith in the solid team and company backing the language.
I'm quite interested in the two mitigations you mentioned. Especially the "manage memory on your own option" ? I was not aware that there was a go language (that is, not a C extension) blessed way to do such a thing.
As for "reduce/avoid" creating garbage, how can I achieve that for a large heap caching application (like the one we're talking about here) ?
In any GC'd language, the less you allocate the less you GC, so "recycling" old objects when you need new ones can help. So where you'd normally make() a []byte, you can instead get one from a list or buffered channel you made of old discarded []bytes. So that gets you back to malloc/free/new/delete, but you can at least do it only for the most performance-relevant allocations in your program and keep being lazy everywhere else.
Another sometimes-useful way to save allocations--not for a cache, but in general--is just to turn short-lived allocations into longer-lived ones: if fooBuf is used and thrown away by each of several calls to obj.baz(), then make a single fooBuf and store it as a (private) field on obj, assuming that doesn't present thread-safety or other problems in the specific context.
There are certainly things you wouldn't use Go for, but now you know more about reducing memory pressure in GC'd languages. :)
Well various techniques to recycle/reuse allocations instead of "garbaging" are well known techniques from the Java world too. Also, the generational collectors popular in the JVM's also do a good job of lots of short-lived allocations (as long as a vast majority of them die young).
The problem is, none of those things help when you have a large cache of small(ish) objects that need to be scanned fully at every GC cycle. There's nothing worse for your performance than stopping the world for a few seconds and using that time to wipe your L1/2/3 caches with completely useless data. That's the reason I called large-heaped caches the anti-pattern for a GC'd system.
Brad's answer about using large byte array allocations (and presumably, managing the smaller chunks manually) is a fair enough answer to this question. But of course in this case you'll be writing code to manage small allocations out of that big buffer yourself, thus negating the whole point of GC. But it might be a decent enough compromise if you like the rest of the language a lot (which Brad clearly does :-)
The easiest way would be to store everything you can on the stack. The Go compiler does a very good job at escape analysis and tries to put everything on the stack that does not escape. Therefore, it's often better to use output parameters instead of return values to reduce (or eliminate) the number of allocations.
Some careful thoughts about the memory layout of your structs, especially which of them should be embedded and / or passed around by pointers and which of them shouldn't, might also pay off.
Another common optimization is to put the allocated objects back to a memory pool for later use. Take a look at the bufCache channel [1] from the bufio package for example (the http and the json package are using the same trick).
I was thinking about that too, but since it is not possible to change existing items and the computation of those items must be re-entrant (groupcache just tries to avoid duplicate computations but does not guarantee it), there seems to be no reason for any distributed consensus. In fact, the groupcache design is astonishing simple.
Yes, Roger Peppe, one of the Go contributors, had exactly the same issue, but luckily Brad and Russ, two core devs, were able to solve the mystery immediately and updated the documentation.
It's now quite funny to read a blog post from someone completely different (Tim Penhey) who had encountered and solved the same issue with exactly the same usecase on the very same day, complaining about Go without even mentioning the discussion [1] or the latest changeset [2] that solved the problem.
Roger Peppe and Tim Penhey both work on Juju at Canonical. One of them wrote to the mail list regarding the issue that they encountered. The other vented his frustration on his personal blog.
Its solves the problem that side effects that are not expected from the documentation of the behavior of the function are produced.
It doesn't solve the "Go interface types specify minimum functionality that must be satisfied at compile time but do not limit the methods that can be called on an object received through the interface type at runtime" problem, but then, that's a fairly fundamental Go design decision, not a problem. (Especially in the context of Go 1.x, which has a no-backward-incompatible-changes commitment.)
The claim in the article, and I agree, and the Go maintainers agree, that it is a problem. It will never be fixed, but it is a problem (or, to put it another way, it causes serious maintainability and understandability problems, if this kind of behavior is common across API boundaries).
I also want to point out that I don't think typecasting itself is a bad thing in all cases. But you shouldn't type cast objects that don't belong to you.
> The claim in the article, and I agree, and the Go maintainers agree, that it is a problem.
I don't think that's accurate. What the Go maintainers seem to agree is that this particular use of the capability to use run-time interface querying on the received value to exceed the capabilities of the interface through which the object is received is problematic because it isn't using an interface of similar semantic role, and this is the kind of thing that would likely be fixed now but for the commitment not to make breaking changes in Go 1.x.
They do not appear to agree that "Go interface types specify minimum functionality that must be satisfied at compile time but do not limit the methods that can be called on an object received through the interface type at runtime" is a problem, just a feature that, while provide a valuable benefit, can also be used in less-than-ideal ways (of which this is an example.)
> But you shouldn't type cast objects that don't belong to you.
On this point in particular, I see no evidence that the Go maintainers agree.
That is pretty funny, but it doesn't really change anything. It's unfortunate that it called Close() without at least giving proper notice, and most people will probably agree that in a perfect world where we could start over without breaking APIs, the implementation would be different.
I was just criticizing the publishing skills of some people... The problem itself is unfortunate and I am personally not a huge fan of the "no API changes" philosophy participated by the Go team. I can understand the reasons behind it, but it's still hard to read several CLs per day that are abandoned or done differently (in a non-optimal way) because of this contract.
The end user gets to decide when they upgrade the toolchain so it wouldn't be "no warning" if they wrote about the change in the release notes. I'd much rather they break code relying on undocumented weird behavior, than document the weird behavior and keep it alive indefinitely.
If we let small semantic changes like this in, it would be too onerous to upgrade, and so nobody would upgrade. Go users should not have to read and understand all these tiny changes, look through their code to see if anything breaks, and finally make the correct fix.
It's just not tenable, and we know this from experience (before Go 1 we introduced breaking changes with each stable release).
FWIW, this is the first time we've encountered this bug, and it's been part of Go since it was released. Hardly worth breaking people's code for.
It's not always obvious if a semantic change that's documented in the release notes will break your code. You might not know you use the code, the particular behaviour might not be covered by your test suite, etc. I like the fact I can upgrade to the latest Go version without having to worry things like that will happen.
That might be right - in this instance - and the actual cost might (or might not) be low - in this instance, but that's not worth breaking the promise Go has made to developers...that code remains compatible across minor forward versions.
