Clustered file systems

Clustered File Systems on Linux

R&D today. There are a lot of filesystems out there for clusters.
So the journey begins for the real truth on CFS (Clustered File Systems)

The idea behind clusters is to join the computing powers of the nodes involved to provide more combined computing power, and/or to build in redundancy for high availability.

All clusters basically fall into two broad categories:
1.) High availability
2.) High-performance computing

High availability clusters provide:
Reliable services, focusing on rapid recovery methods in case of component or system failure, protecting applications and data

High-performance computing:
Provides greater computational power through compute clusters than one computer alone could

The core of any cluster solution is a clustered file system and optional a resource manager for failover.
A list with all Linux filesystems

High Performace Computing (HPC)
If you seek for a High-Performance Computing solution, involving clustering multiple processors for intensive computation, Lustre is the best option.

With SLES9 Lustre Lite is included. see features
What's so 'lite' about it?
Cluster File Systems Inc themselves call the currently shipping versions
"Lustre Lite", because even though it scales to petabytes, it is only a
subset of the intergalactic filesystem they eventually want to build.
Talk about ambitions!
Lustre is a HPC distributed filesystem and not applicable to your
database server at all.
It's an asymetric solution that stores it's info in metadataservers
Better for read operations?
Lustre is aimed at the HPC space. Lustre is a client-server distributed
filesystem architecture, where a set of Highly Available servers serves
tenth to tenth of thousands of clients. Storage is local to the servers
and accessed by the clients through the Lustre network protocol from
the servers. Lustre is complex to setup and does not offer a builtin HA
solution.
Worlds largest Linux based HPC environments run on SUSE with Lustre.
Some weather forecast organisations moved their app. from SUN machines with tons
of processors to low cost intel hardware (blades) with SUSE Linux and Lustre.

RedHat positions (Sistina) GFS in this space and Polyserve Matrix Server is an option.
Lustre is designed for this work, and it's open source.
Lustre support +/- 10000 nodes and can be hard to setup.
Used in Nat Labs and Academics Institutions environments.
HP StorageWorks Scalable File Share based on Lustre

File and Print Services (office automation)
The most advanced file system for office automation is NSS (Novell Storage Service) from Novell. With Novell Open Enterprise Server the NSS filesystem is available on Linux. Novell Cluster Services (NCS) is part of OES and adds a resource failover solution for any network service. NSS has support for 32 nodes and can be used in combination with other filesystems as well. NCS in combination with NSS is ideal for large office automation environments, including other netwerk services like messaging, webservers and others. (no database apps)
BCC (Business Continuity Clusters) can expand the solution to a GEO solution

File Services (heavy)
For some customers a single server that hosts a volume is not enough. Think of heavy load of filesystems, mulitmedia read and write operations, the real large files. This has diffrent requirements than an office automation environment. The best solution here is a Polyserve Matrix server with the file serving solution pack. This allows multiple server to share the same volume for read and write operations.

Unix to Linux migration (Oracle apps)
Scalability and high availability are requirements is you talk about datacentre application migration to Linux. A rock solid clustered file system and a good resource manager are essential.

Polyserve Matrix Server with the database solution pack gives you real symetric scalability.
Polyserve does not require a metadata server.

Another option is to add the Novell Polyserve Matrix Server to your SLES.
OEM from Polyserv
Article: Why Novell ships Polyserve

PolyServe's cluster file system is a Storage Area Network concurrent
access cluster file system geared at the data center, and capable of
supporting an Oracle shared home for example.
Polyserve is a symetric solution that does not rely on metadataservers.

PolyServe MatrixCluster is a commercial solution, providing a
concurrent access cluster file system, based on a Storage Area Network
in the background; supporting up to 16 nodes. PolyServe is symmetric,
and does not require a client/server architecture. PolyServe has a 16
node HA solution too. PolyServe is NOT Open Source.

RedHat (Old Sistina GFS) GFS project - Global File System
300 GFS nodes as max?

Oracle Cluster File System v2 (OCFS2) on SLES (OCFS2 = open source)
Because OCFSv2 is quite new most Oracle customer (today) use other solutions, like Polyserv or GFS. However Polyserve states that using it's clustering tools in stead of Oracles
has benefits. Maybe performance - maybe price - check the website.
Novell / SUSE will adopt OCFS v2.
OCFS v2 ships with SUSE Enterprise Server 9 and later.

Conclusion
HPC solution, Lustre seems the best solution in the industry.
Lustre is what one should use for HPC workloads.

For database solution like Oracle, there is ocfs v2, gfs and Polyserve Matrix server
Polyserve looks like the most advanced solution. GFS is opensource but is not free ($2200,- per server). OCFS v2 will be the best solution over time. (today not a lot of customers)
Polyserve seems the best solution on the market today for Oracle

RedHat vs Novell/SuSE
Both Novel/SUSE invest a lot of development power in OCFS v2.
The only logic choice on RedHat is GFS. GFS is not supported on SUSE.
Polyserve Matrix server is supported on both RedHat and SUSE.

If you do Oracle, look at OCFS. Oracle made a strategic parnership with Novel and
does their internal development on SUSE these days. Going forward Oracle, ocfs and suse
is the most logic path. GFS will give you vendor lock-in with RedHat. GFS will only be available on RedHat. OCFS is available SUSE today but RedHat needs to support OCFS.

Office automation (high availability)
Novell has its roots in file services. The Novell file systems in combination with NCS looks like the best solution for a normal office environment with clusters for h.a. on Linux.

heavy file serving (nfs)
Polyserve can also be used for shared disk systems (scalable file serving
& common config files for webservers), and competes against RedHat GFS
(and does a much better job than GFS)


keywords: ocfs, ocfsv2, nfs, cifs, oracle rac, oracle 10g, cluster, novell, lustre, red hat, gfs, veritas