Distributed File System

  • Tech Stack: C · System Calls · Operating System · Networks
  • Project URL: Github Link

Naming and Storage Servers

1. Initialization

Naming Server (NM)

  • Set up to manage directory structures and file locations.
  • Solution: Define a port for NM in header.h. NM listens on this port.

Storage Servers (SS_1 to SS_n)

SS Initialization

  • SS_1 Initialization:
    • Provide NM with IP, client connection ports, and accessible paths.
    • Solution: SS registers with NM, sending its IP, port, and directory details.
  • SS_2 to SS_n Initialization:
    • Follow the same procedure as SS_1.

NM Client Requests

  • Begin accepting client requests after all SS are initialized.
  • Solution: Clients connect to NM with an ID to establish a connection.

2. On Storage Servers (SS)

Adding New Storage Servers

  • Dynamically add entries to NM.
  • Solution: NM uses SELECT to accept new connections from SS and clients.

Commands Issued by NM

  • Create File/Directory: SS receives the path and command (CREATE or CREATE_D) and sends an ACK.
  • Delete File/Directory: SS receives the path and command (DELETE) and sends an ACK.
  • Copy Files:
    • One SS reads data, sends it to NM, which forwards it to another SS if needed. SS creates the file and sends an ACK.
  • Copy Directory:
    • SS sends paths to NM, NM instructs another SS to recreate the structure, and NM handles file copying. The receiving SS sends an ACK to NM and NM to the client.

Client Interactions

  • Read a File: NM provides the client the SS's IP and port; the client requests the file from SS.
  • Write to a File: NM provides the client the SS's IP and port; the client sends the data to SS to write.
  • Get Size and Permissions: NM provides the SS's IP and port; the client requests info from SS.

3. On Naming Server (NM)

Storing Storage Server Data

  • Serve as the central repository for SS information and paths.

Client Task Feedback

  • NM sends an ACK to clients upon task completion.

Clients

Client-NM Communication

  • Clients connect to NM with a unique ID and perform various commands.

Functionalities

Read, Write, Retrieve Info

  • NM searches cache/trie and provides SS details to the client for direct communication.

Create/Delete Files/Folders

  • NM searches cache/trie, sends tasks to SS, receives ACK, and forwards to client.

Copy Files/Directories

  • NM handles paths, communicates with SS for tasks, and manages copying, sending an ACK to the client.

Other Features

Multiple Clients

Concurrent Access

  • Use select system call for handling multiple clients.
  • Clients can perform multiple operations, and new clients can connect anytime.

Error Codes

  • Define and send clear error codes for various failures.
#define ERROR_READ 100
#define ERROR_WRITE 101
#define ERROR_CREATE 102
#define ERROR_CREATE_D 103
#define ERROR_DELETE 104
#define ERROR_COPY 105
#define ERROR_INFO 106

Redundancy/Replication

Failure Detection

  • NM detects SS failures and logs them.

Data Redundancy

  • Duplicate files and folders in two other SS to allow read operations during failures.

SS Recovery

  • Restore original SS from duplicates, preventing new entries during recovery.

Asynchronous Duplication

  • Write commands are duplicated asynchronously.

Bookkeeping

Logging

  • Record every request and acknowledgment with relevant details.

Message Display

  • Display messages in NM and SS for operations.

IP and Port Recording

  • Log IP addresses and ports for traceability.