rewriite

README.md

@@ -1,12 +1,23 @@
 # go-cdn
 
-An experimental CDN project in Go.
+> An experimental CDN project in Go.
+> A lightweight server that allows you to upload and download files via HTTP. It provides optional authentication, input validation, error handling, and verbose logging. The server is implemented in Go and can be easily configured using a config.json file.
 
 ## Features
 
 - Serve static files securely over HTTP
 - Basic authentication support
-- File upload functionality
+- File upload & download functionality
+
+### Error Handling
+
+The server is equipped with error handling to handle various scenarios, including invalid requests, authentication failures, and payload size exceeding the maximum allowed limit. If an error occurs, the server will return an appropriate HTTP status code along with an error message.
+Logging
+
+The server uses the logrus library for logging. Verbose logging is enabled by default and displays detailed information about incoming requests, errors, and server restarts. The logs are printed to the console.
+Customization
+
+The server implementation provided is a starting point and may require customization based on your specific requirements. You can modify the handleGet and handlePost functions in the CDNHandler struct to implement your desired file upload and download logic.
 
 ## Possible use cases
 
@@ -18,64 +29,107 @@ The CDN can be used in various scenarios, such as:
 
 ## Usage
 
-### Starting the CDN
+### Prerequisites
 
-1. Build the program
+- Go programming language (version 1.16 or later)
 
-```go
+### Installation
-go build
-```
 
-2. Run the built binary
+1. Clone the repository:
-
-```go
-
-```
-
-### Using the CDN
-
-To request a file from the CDN, use the following URL format:
 
 ```bash
-http://your-cdn-server:port/file-path
+
+git clone https://github.com/notashelf/go-cdn.git
 ```
 
-> Replace your-cdn-server with the hostname or IP address of your CDN server and port with the port number on which the server is running. Append the desired file path after the hostname and port.
+2. Change to the project directory:
-
-**Example:**
-
-- `http://localhost:8080/images/file_you_have_uploaded.png`
-
-  - If the file exists and the request is authorized, the file will be served by the CDN.
-  - If the file doesn't exist, a 404 Not Found response will be returned.
-
-#### Uploading Files to the CDN
-
-> Send a POST request to the /upload endpoint of the CDN server.
-
-Example using cURL:
 
 ```bash
-curl -X POST -u username:password -F "file=@/path/to/file" http://your-cdn-server:port/upload
+cd your-repo
 ```
 
-> Replace username and password with the authentication credentials you have set in the main.go file. Replace your-cdn-server with the hostname or IP address of your CDN server and port with the port number on which the server is running. Provide the file path after the @ symbol in the -F parameter.
+3. Install the dependencies:
-
-**Example:**
 
 ```bash
-curl -X POST -u admin:password -F "file=@/absolute/path/to/image.jpg" http://localhost:8080/upload
+go get github.com/sirupsen/logrus
+go get github.com/pkg/errors
 ```
 
-    - The uploaded file will be saved in the specified uploadPath directory.
+### Configuration
 
-> Note: The server responds with a success message if the file upload is successful.
+The server can be configured using the config.json file. Create a file named config.json in the same directory as the main file (main.go) and configure the following properties:
 
-### Security Considerations
+- port (string): The port on which the server will listen for incoming connections.
+- max_upload_size (integer): The maximum allowed size of file uploads, in bytes.
+- heartbeat (string): The duration after which the server will automatically restart. Specify a value in the format "5m" for 5 minutes, "1h" for 1 hour, etc. Set to "0" to disable automatic restarts.
+- require_auth (boolean): Whether to require authentication for file uploads and downloads.
+- auth_username (string): The username for authentication (only applicable if require_auth is set to true).
+- auth_password (string): The password for authentication (only applicable if require_auth is set to true).
 
-It is highly recommended to use SSL/TLS encryption (HTTPS) for secure communication between clients and the CDN server.
+Example config.json file:
-Change the default username and password in the main.go file to strong and secure credentials.
+
-Consider implementing additional security measures based on your specific requirements.
+```json
+{
+  "port": "8080",
+  "max_upload_size": 10485760,
+  "heartbeat": "1h",
+  "require_auth": true,
+  "auth_username": "your-username",
+  "auth_password": "your-password"
+}
+```
+
+### Usage
+
+Start the CDN server by running the following command:
+
+```bash
+go run main.go -config config.json
+```
+
+The server will start and listen on the specified port. Verbose log messages will be displayed in the console.
+
+#### Uploading a file:
+
+Use the curl command to upload a file to the CDN server:
+
+```bash
+curl -X POST -F "file=@/path/to/file" http://localhost:8080
+```
+
+_Replace /path/to/file with the actual path to the file you want to upload. If authentication is enabled, provide the username and password when prompted._
+
+#### Downloading a file:
+
+Use the curl command to download a file from the CDN server:
+
+```bash
+curl -O http://localhost:8080/<filename>
+```
+
+_Replace <filename> with the name of the file you want to download._
+
+### Authenticated Upload and Download
+
+To perform an authenticated upload or download, you can use the following curl commands:
+
+#### Uploading a File:
+
+```bash
+curl -X POST -F "file=@/path/to/file" -u "your-username:your-password" http://localhost:8080
+```
+
+_Replace /path/to/file with the actual path to the file you want to upload. The -u flag is used to provide the authentication credentials._
+
+#### Downloading a File:
+
+```bash
+curl -O -u "your-username:your-password" http://localhost:8080/<filename>
+```
+
+Replace <filename> with the name of the file you want to download. The -u flag is used to provide the authentication credentials.
+
+**Please note that these examples assume you're running the server on localhost with the specified port and authentication credentials. Make sure to adjust the hostname and port accordingly.**
 
 # License
 

example_config.json

@@ -1,6 +0,0 @@
-{
-  "username": "admin",
-  "password": "password",
-  "service_port": "8080",
-  "uploads_dir": "uploads/"
-}

go.mod

@@ -3,7 +3,7 @@ module cdn
 go 1.20
 
 require (
-	github.com/go-chi/chi v1.5.4 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
 	github.com/sirupsen/logrus v1.9.2 // indirect
-	golang.org/x/sys v0.8.0 // indirect
+	golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 // indirect
 )

go.sum

@@ -1,14 +1,13 @@
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/go-chi/chi v1.5.4 h1:QHdzF2szwjqVV4wmByUnTcsbIg7UGaQ0tPF2t5GcAIs=
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
-github.com/go-chi/chi v1.5.4/go.mod h1:uaf8YgoFazUOkPBG7fxPftUylNumIev9awIWOENIuEg=
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/sirupsen/logrus v1.9.2 h1:oxx1eChJGI6Uks2ZC4W1zpLlVgqB8ner4EuQwV4Ik1Y=
 github.com/sirupsen/logrus v1.9.2/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 h1:0A+M6Uqn+Eje4kHMK80dtF3JCXC4ykBgQG4Fe06QRhQ=
 golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU=
-golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

main.go

@@ -2,236 +2,205 @@ package main
 
 import (
 	"encoding/json"
+	"flag"
 	"fmt"
 	"io"
+	"log"
 	"net/http"
 	"os"
 	"path/filepath"
-	"regexp"
+	"time"
-	"runtime"
-	"strings"
 
-	"github.com/go-chi/chi"
-	"github.com/go-chi/chi/middleware"
 	"github.com/sirupsen/logrus"
 )
 
+// Config represents the configuration structure
 type Config struct {
-	Username    string `json:"username"`
+	Port            string   `json:"port"`
-	Password    string `json:"password"`
+	MaxUploadSize   int64    `json:"max_upload_size"`
-	ServicePort string `json:"service_port"`
+	Heartbeat       Duration `json:"heartbeat"`
-	UploadsDir  string `json:"uploads_dir"`
+	RequireAuth     bool     `json:"require_auth"`
+	AuthUsername    string   `json:"auth_username"`
+	AuthPassword    string   `json:"auth_password"`
+	UploadDirectory string   `json:"upload_directory"`
 }
 
-var (
+// Duration is a custom type for decoding time.Duration from JSON
-	config        Config
+type Duration time.Duration
-	logger        *logrus.Logger
-	filenameRegex *regexp.Regexp
-)
 
-func main() {
+// UnmarshalJSON unmarshals a JSON string into a Duration
-	// Initialize logger
+func (d *Duration) UnmarshalJSON(data []byte) error {
-	logger = logrus.New()
+	var durationStr string
-	logger.Formatter = &logrus.TextFormatter{
+	if err := json.Unmarshal(data, &durationStr); err != nil {
-		DisableTimestamp: false,
+		return fmt.Errorf("error decoding Duration: %w", err)
-		FullTimestamp:    true,
 	}
 
-	// Load configuration
+	parsedDuration, err := time.ParseDuration(durationStr)
-	err := loadConfig("config.json")
 	if err != nil {
-		logger.Fatalf("Error loading config file: %s", err)
+		return fmt.Errorf("error parsing Duration: %w", err)
-	}
-
-	// Initialize filename regular expression
-	filenameRegex = regexp.MustCompile(`^[a-zA-Z0-9-_\.]+$`)
-
-	// Initialize router
-	router := chi.NewRouter()
-	router.Use(middleware.Logger)
-
-	router.HandleFunc("/", serveCDN)
-	router.HandleFunc("/upload", handleUpload)
-
-	// Start server
-	address := fmt.Sprintf(":%s", config.ServicePort)
-	logger.Infof("Starting CDN server on port %s...", config.ServicePort)
-	logger.Infof("Serving files from %s", config.UploadsDir)
-
-	// Print upload path and list files
-	uploadPath := filepath.Join(".", config.UploadsDir)
-	logger.Infof("Upload path: %s", uploadPath)
-	listFiles(uploadPath)
-
-	err = http.ListenAndServe(address, router)
-	if err != nil {
-		logger.Fatalf("Server error: %s", err)
-	}
-}
-
-func loadConfig(filename string) error {
-	// Get the absolute path of the main.go file
-	_, currentFile, _, _ := runtime.Caller(1)
-	currentDir := filepath.Dir(currentFile)
-
-	// Construct the absolute path for the config file
-	configPath := filepath.Join(currentDir, filename)
-
-	// Open the config file
-	file, err := os.Open(configPath)
-	if err != nil {
-		return fmt.Errorf("error opening config file: %s", err)
-	}
-	defer file.Close()
-
-	// Decode the config file into the config variable
-	decoder := json.NewDecoder(file)
-	err = decoder.Decode(&config)
-	if err != nil {
-		return fmt.Errorf("error decoding config file: %s", err)
-	}
-
-	// Set the relative path for the uploads directory
-	config.UploadsDir = filepath.Join(currentDir, filepath.FromSlash(config.UploadsDir))
-
-	// Create the uploads directory if it doesn't exist
-	if _, err := os.Stat(config.UploadsDir); os.IsNotExist(err) {
-		err := os.MkdirAll(config.UploadsDir, os.ModePerm)
-		if err != nil {
-			return fmt.Errorf("error creating uploads directory: %s", err)
-		}
 	}
 
+	*d = Duration(parsedDuration)
 	return nil
 }
 
-func serveCDN(w http.ResponseWriter, r *http.Request) {
+// CDNHandler handles HTTP requests to the CDN server
-	// Log the request information
+type CDNHandler struct {
-	logger.Infof("Received request: %s %s", r.Method, r.URL.Path)
+	Config Config
-
+	Logger *logrus.Logger
-	// Check if the request has valid authentication
-	if !checkAuthentication(r) {
-		logger.Warn("Authentication failed")
-		w.Header().Set("WWW-Authenticate", `Basic realm="CDN Authentication"`)
-		w.WriteHeader(http.StatusUnauthorized)
-		fmt.Fprintf(w, "401 Unauthorized\n")
-		return
 }
 
-	// Get the requested file path
+// ServeHTTP serves HTTP requests
-	filePath := filepath.Join(config.UploadsDir, filepath.Clean(r.URL.Path))
+func (c *CDNHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	switch r.Method {
+	case http.MethodGet:
+		c.handleGet(w, r)
+	case http.MethodPost:
+		c.handlePost(w, r)
+	default:
+		http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+	}
+}
 
-	// Check if the file exists
+// handleGet handles GET requests
-	_, err := os.Stat(filePath)
+func (c *CDNHandler) handleGet(w http.ResponseWriter, r *http.Request) {
+	c.Logger.Infof("Received GET request for URL: %s", r.URL.Path)
+
+	// Serve file for download
+	filePath := filepath.Join(c.Config.UploadDirectory, r.URL.Path)
+	file, err := os.Open(filePath)
 	if err != nil {
-		if os.IsNotExist(err) {
+		c.Logger.Errorf("Error opening file: %v", err)
-			logger.Infof("File not found: %s", r.URL.Path)
+		http.Error(w, "File Not Found", http.StatusNotFound)
-			http.NotFound(w, r)
+		return
-		} else {
+	}
-			logger.Error("Internal Server Error:", err)
+	defer file.Close()
+
+	// Set the appropriate Content-Type header based on file extension
+	contentType := "application/octet-stream"
+	switch filepath.Ext(filePath) {
+	case ".jpg", ".jpeg":
+		contentType = "image/jpeg"
+	case ".png":
+		contentType = "image/png"
+	case ".pdf":
+		contentType = "application/pdf"
+	}
+
+	w.Header().Set("Content-Type", contentType)
+	if _, err := io.Copy(w, file); err != nil {
+		c.Logger.Errorf("Error copying file: %v", err)
 		http.Error(w, "Internal Server Error", http.StatusInternalServerError)
-		}
 		return
 	}
 
-	// Serve the file
+	c.Logger.Infof("File downloaded successfully: %s", r.URL.Path)
-	logger.Infof("Serving file: %s", r.URL.Path)
-	http.ServeFile(w, r, filePath)
 }
 
-func handleUpload(w http.ResponseWriter, r *http.Request) {
+// handlePost handles POST requests
-	// Log the request information
+func (c *CDNHandler) handlePost(w http.ResponseWriter, r *http.Request) {
-	logger.Infof("Received upload request: %s %s", r.Method, r.URL.Path)
+	c.Logger.Info("Received POST request")
 
-	// Check if the request has valid authentication
+	// Validate request size
-	if !checkAuthentication(r) {
+	r.Body = http.MaxBytesReader(w, r.Body, c.Config.MaxUploadSize)
-		logger.Warn("Authentication failed")
+	if err := r.ParseMultipartForm(c.Config.MaxUploadSize); err != nil {
-		w.Header().Set("WWW-Authenticate", `Basic realm="CDN Authentication"`)
+		c.Logger.Errorf("Error parsing multipart form: %v", err)
-		w.WriteHeader(http.StatusUnauthorized)
+		http.Error(w, "Payload Too Large", http.StatusRequestEntityTooLarge)
-		fmt.Fprintf(w, "401 Unauthorized\n")
-		return
-	}
-
-	// Parse the uploaded file
-	err := r.ParseMultipartForm(32 << 20) // Max file size: 32MB
-	if err != nil {
-		logger.Error("Bad Request:", err)
-		http.Error(w, "Bad Request", http.StatusBadRequest)
 		return
 	}
 
+	// Get the uploaded file
 	file, handler, err := r.FormFile("file")
 	if err != nil {
-		logger.Error("No file provided in the request:", err)
+		c.Logger.Errorf("Error retrieving file: %v", err)
-		http.Error(w, "No file provided in the request", http.StatusBadRequest)
+		http.Error(w, "Error retrieving file", http.StatusInternalServerError)
 		return
 	}
 	defer file.Close()
 
-	// Validate filename
+	// Create the upload directory if it doesn't exist
-	filename := sanitizeFilename(handler.Filename)
+	uploadDir := c.Config.UploadDirectory
-	if !isValidFilename(filename) {
+	if uploadDir == "" {
-		logger.Errorf("Invalid filename: %s", handler.Filename)
+		uploadDir = "uploads"
-		http.Error(w, "Invalid filename", http.StatusBadRequest)
+	}
+	if err := os.MkdirAll(uploadDir, os.ModePerm); err != nil {
+		c.Logger.Errorf("Error creating upload directory: %v", err)
+		http.Error(w, "Error creating upload directory", http.StatusInternalServerError)
 		return
 	}
 
-	// Create the uploads directory if it doesn't exist
+	// Create the file in the upload directory
-	err = os.MkdirAll(config.UploadsDir, os.ModePerm)
+	filePath := filepath.Join(uploadDir, handler.Filename)
+	newFile, err := os.Create(filePath)
 	if err != nil {
-		logger.Error("Error creating uploads directory:", err)
+		c.Logger.Errorf("Error creating file: %v", err)
-		http.Error(w, "Error creating uploads directory", http.StatusInternalServerError)
+		http.Error(w, "Error creating file", http.StatusInternalServerError)
+		return
+	}
+	defer newFile.Close()
+
+	// Copy the uploaded file to the new file
+	if _, err := io.Copy(newFile, file); err != nil {
+		c.Logger.Errorf("Error copying file: %v", err)
+		http.Error(w, "Error copying file", http.StatusInternalServerError)
 		return
 	}
 
-	// Create a new file in the uploads directory
+	c.Logger.Infof("File uploaded successfully: %s", handler.Filename)
-	dst, err := os.Create(filepath.Join(config.UploadsDir, filename))
+	fmt.Fprint(w, "File uploaded successfully")
-	if err != nil {
-		logger.Error("Internal Server Error:", err)
-		http.Error(w, "Internal Server Error", http.StatusInternalServerError)
-		return
-	}
-	defer dst.Close()
-
-	// Copy the uploaded file to the destination
-	_, err = io.Copy(dst, file)
-	if err != nil {
-		logger.Error("Internal Server Error:", err)
-		http.Error(w, "Internal Server Error", http.StatusInternalServerError)
-		return
 }
 
-	logger.Infof("File uploaded successfully: %s", filename)
+func main() {
-	fmt.Fprintf(w, "File uploaded successfully!")
+	// Parse command line flags
-}
+	configPath := flag.String("config", "config.json", "Path to the configuration file")
-func checkAuthentication(r *http.Request) bool {
+	flag.Parse()
-	username, password, ok := r.BasicAuth()
-	return ok && username == config.Username && password == config.Password
-}
 
-func sanitizeFilename(filename string) string {
+	// Initialize logrus logger
-	return strings.TrimSpace(filename)
+	logger := logrus.New()
-}
+	logger.SetFormatter(&logrus.TextFormatter{
-
+		FullTimestamp:   true,
-func isValidFilename(filename string) bool {
+		TimestampFormat: "2006-01-02 15:04:05",
-	return filenameRegex.MatchString(filename)
-}
-
-func listFiles(dirPath string) {
-	logger.Infof("Files in %s:", dirPath)
-
-	err := filepath.Walk(dirPath, func(path string, info os.FileInfo, err error) error {
-		if err != nil {
-			logger.Errorf("Error accessing file: %s", err)
-			return nil
-		}
-		if !info.IsDir() {
-			logger.Infof("- %s", path)
-		}
-		return nil
 	})
 
+	// Read the configuration file
+	configFile, err := os.Open(*configPath)
 	if err != nil {
-		logger.Errorf("Error listing files: %s", err)
+		logger.Fatalf("Error opening configuration file: %v", err)
 	}
+	defer configFile.Close()
+
+	// Decode the configuration file
+	var config Config
+	err = json.NewDecoder(configFile).Decode(&config)
+	if err != nil {
+		logger.Fatalf("Error decoding configuration file: %v", err)
+	}
+
+	// Start a goroutine to restart the server periodically
+	if config.Heartbeat > 0 {
+		go func() {
+			for range time.Tick(time.Duration(config.Heartbeat)) {
+				logger.Info("Server heartbeat")
+				os.Exit(0)
+			}
+		}()
+	}
+
+	// Create a new CDNHandler with the configuration
+	cdnHandler := &CDNHandler{
+		Config: config,
+		Logger: logger,
+	}
+
+	// Create a new HTTP server
+	server := &http.Server{
+		Addr:         ":" + config.Port,
+		Handler:      cdnHandler,
+		ErrorLog:     log.New(logger.Writer(), "", 0),
+		ReadTimeout:  5 * time.Second,
+		WriteTimeout: 10 * time.Second,
+	}
+
+	logger.Infof("Starting CDN server on port %s", config.Port)
+	log.Fatal(server.ListenAndServe())
 }