File System Features Support (Under the hood)

This article is part of our "Tiger Bridge - Under the hood" series, where we uncover technical details about our product, dive deep into the solutions it offers, and explain which decisions we had to make while designing it.

Dilemma

We had to decide whether to integrate in the existing file system or provide an external one like some other solutions on the market. The data path from the application user through the operating system to the file system and eventually the storage underneath goes like this:

So it was a matter of figuring out where we wanted to implement our solution: before or after the operating system in that data flow.

Background

A common approach is to create cross-platform solutions which provide universality but often sacrifice functionality and speed.

Historically speaking, before the cloud was adopted on a broader scale, the alternative to using local storage were network shares. With this approach, the operating system connects to a net server to get the needed data.

Using a web server to host the files could be another approach. In this scenario, however, the operating system cannot communicate with the web server – only the applications can. In other words, the connection goes from the application to the web server while still carrying the same purpose.

In cloud storage, there is no file system – we have a web server using the proprietary operating system to connect to object storage, utilizing some storage infrastructure below. Therefore, data needs to be accessed through the application again.

Instead of relying on Windows Explorer to reach your files, you may opt for the browser. This could give you additional benefits of using the cloud. However, it comes with sacrificing your normal workflow. For certain applications, there is no option for that. If they are not designed to support such a change, using the cloud becomes impossible. Since the user would still communicate with the application, we had to place our solution in such a way so it could become transparent for both the user and the application.

The question was whether to implement our software before or after the operating system in the data flow diagram.

The lower its position in the data flow, the slower the solution becomes. Even the smallest delay on a lower level grows significantly after being multiplied millions of times for the higher levels’ operations. This is why we need to be at the highest position while not sacrificing the functionality of our clients’ applications. Implementation costs and time also grow substantially the lower you get in the data flow.

Many other solutions use external file systems which are already implemented, like Fuze. Just by integrating a few callbacks, the solutions get a system for Windows, Linux, Mac, or any other OS. In order to work with a file system which supports all OSs, Fuze needs to take the limitations of all of them and provide these systems with common functions.

To support all operating systems, Fuze removes every advanced function, essentially cutting support for it. As previously discussed, for some applications, these functions may be critical. The applications would fail if the functions are not supported by the file system.

We serve clients who cannot afford to lose even a single millisecond hence we considered a lower implementation option:

Pros/Cons Analysis

While making our decision, we considered the following for the lower implementation scenario:

Decision

We concluded that the external (on top of the OS) file system is not a good option for us and our clients. We chose the lower option (between the OS and the file system).

Arguments

Big cloud providers such as Microsoft, Amazon and Google have decided to walk on the easier and faster path of upper-level implementation. We understand their decision.

The lower in the data flow we reside, the bigger the implementation control we have. This way, we can integrate functions which are unavailable on the higher level. It all comes with lower control of the overall workflow, resulting in higher implementation and support cost. As is the case with other decisions, we choose to make our lives harder in favor of clients and their applications’ needs.

We cannot support third-party tools and therefore have to build individual platforms for the different OSs, which is something we are ready to do.

Residing lower in the data flow means there are more rules we need to account for. However, investment for multiple years and previous experience helped us build a much faster lower-level solution. The implementation cost of time and money was not as big as it might be for other providers.

Conclusion

As always, we are fully devoted to our clients. The full control we get with our decision for direct file system support allows us to resolve corner cases which have been show-stoppers for cloud adoption.