Cloudflare R2 combines friendly pricing—no egress fees—with an S3-compatible API, making it easy to integrate with existing tooling. In this DevTip, you’ll learn how to push files to R2 from Scala using the AWS SDK v2, Cats Effect for asynchronous operations, and FS2 for streaming.

Why choose Cloudflare R2?

R2 is object storage that speaks the same language as Amazon S3 while charging zero for data leaving its buckets (egress). That’s perfect for edge-heavy workloads or public downloads where bandwidth prices quickly add up. Because the API matches S3, you can leverage the robust AWS SDKs.

Set up your Scala project

Add the necessary dependencies to your build.sbt file. We'll use the AWS SDK v2 for S3 interactions, Cats Effect for managing asynchronous operations, and FS2 for file streaming.

// build.sbt
libraryDependencies ++= Seq(
  // AWS SDK v2 for S3 (compatible with R2)
  "software.amazon.awssdk" % "s3" % "2.25.12",
  // Cats Effect for asynchronous programming
  "org.typelevel" %% "cats-effect" % "3.6.1",
  // FS2 for streaming
  "co.fs2" %% "fs2-core" % "3.10.2",
  "co.fs2" %% "fs2-io" % "3.10.2",
  // Bridge FS2 streams to Reactive Streams (for AWS SDK AsyncRequestBody)
  "co.fs2" %% "fs2-reactive-streams" % "3.10.2"
)

Configure the AWS SDK client

You need to configure the S3 client with your R2 credentials and the specific R2 endpoint. R2 requires the region to be set, typically auto, but the SDK might require a specific AWS region like us-east-1 for signing purposes, even though the endpoint points to Cloudflare.

import software.amazon.awssdk.auth.credentials.{AwsBasicCredentials, StaticCredentialsProvider}
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.s3.S3Client
import software.amazon.awssdk.services.s3.S3AsyncClient
import java.net.URI

val accountId   = sys.env("R2_ACCOUNT_ID") // Your Cloudflare Account ID
val accessKey   = sys.env("R2_ACCESS_KEY_ID") // Your R2 Access Key ID
val secretKey   = sys.env("R2_SECRET_ACCESS_KEY") // Your R2 Secret Access Key
val r2Endpoint  = s"https://$accountId.r2.cloudflarestorage.com"

val credentials = AwsBasicCredentials.create(accessKey, secretKey)
val credProvider = StaticCredentialsProvider.create(credentials)

// Synchronous Client
val s3SyncClient: S3Client = S3Client.builder()
  .credentialsProvider(credProvider)
  .region(Region.US_EAST_1) // Use a valid AWS region for signing
  .endpointOverride(URI.create(r2Endpoint))
  .build()

// Asynchronous Client (for Cats Effect / FS2 integration)
val s3AsyncClient: S3AsyncClient = S3AsyncClient.builder()
  .credentialsProvider(credProvider)
  .region(Region.US_EAST_1) // Use a valid AWS region for signing
  .endpointOverride(URI.create(r2Endpoint))
  .build()

Note: Always fetch credentials securely, for example, from environment variables or a secrets manager, not hardcoded in your source code.

Basic synchronous upload

For simple cases or scripts, a synchronous upload is straightforward using the S3Client.

import software.amazon.awssdk.services.s3.model.PutObjectRequest
import software.amazon.awssdk.core.sync.RequestBody
import java.nio.file.Paths

val bucketName = "your-r2-bucket-name"
val fileKey    = "uploads/hello.txt" // The object key (path) in the bucket
val filePath   = Paths.get("./hello.txt") // Local file to upload

val putReq = PutObjectRequest.builder()
  .bucket(bucketName)
  .key(fileKey)
  .contentType("text/plain") // Set appropriate content type
  .build()

try {
  val response = s3SyncClient.putObject(putReq, RequestBody.fromFile(filePath))
  println(s"Upload successful! ETag: ${response.eTag()}")
} catch {
  case e: Exception => println(s"Upload failed: ${e.getMessage}")
}

The AWS SDK automatically handles signing the request (AWS Signature V4) using the provided credentials.

Asynchronous upload with cats effect

For better performance and integration into non-blocking applications, use the S3AsyncClient with Cats Effect. We need a small helper to bridge CompletableFuture (used by the SDK) to IO.

import cats.effect.{IO, IOApp, Outcome}
import software.amazon.awssdk.services.s3.model.PutObjectResponse
import java.util.concurrent.CompletableFuture

// Helper to convert CompletableFuture to IO
def fromCompletableFuture[A](cfa: => CompletableFuture[A]): IO[A] = {
  IO.async_ { cb =>
    IO {
      val cf = cfa
      cf.handle[Unit] { (res, err) =>
        err match {
          case null => cb(Right(res))
          case ex   => cb(Left(ex))
        }
      }
      Some(IO(cf.cancel(true))) // Cancellation logic
    }
  }
}

object AsyncUploader extends IOApp.Simple {
  val bucketName = "your-r2-bucket-name"
  val fileKey    = "uploads/async-hello.txt"
  val filePath   = Paths.get("./hello.txt")

  val putReq = PutObjectRequest.builder()
    .bucket(bucketName)
    .key(fileKey)
    .contentType("text/plain")
    .build()

  def run: IO[Unit] = {
    val uploadIO: IO[PutObjectResponse] = fromCompletableFuture {
      s3AsyncClient.putObject(putReq, RequestBody.fromFile(filePath))
    }

    uploadIO.flatMap(resp => IO.println(s"Async upload successful! ETag: ${resp.eTag()}"))
      .handleErrorWith(err => IO.println(s"Async upload failed: ${err.getMessage}"))
      .guaranteeCase { // Ensure client cleanup
        case Outcome.Succeeded(_) | Outcome.Errored(_) | Outcome.Canceled() =>
          IO(s3AsyncClient.close()) *> IO(s3SyncClient.close()) // Close both if used
      }
  }
}

Handling responses and errors

The AWS SDK throws specific exceptions for different error conditions (e.g., S3Exception, NoSuchBucketException, SdkClientException). You can catch these for fine-grained error handling.

import software.amazon.awssdk.services.s3.model.S3Exception
import software.amazon.awssdk.core.exception.SdkClientException

// Inside your IO error handler or try-catch block
// ...
.handleErrorWith {
  case e: S3Exception if e.statusCode() == 403 =>
    IO.println("Access Denied (403): Check credentials or bucket policy.")
  case e: S3Exception if e.statusCode() == 404 =>
    IO.println("Not Found (404): Check bucket name or object key.")
  case e: S3Exception =>
    IO.println(s"S3 Service Error (${e.statusCode()}): ${e.awsErrorDetails().errorMessage()}")
  case e: SdkClientException =>
    IO.println(s"SDK Client Error (e.g., network issue): ${e.getMessage}")
  case e: Exception =>
    IO.println(s"Generic Error: ${e.getMessage}")
}

Retry logic with exponential back-off

Network glitches or temporary service issues (like rate limiting) happen. Implement a retry mechanism using Cats Effect.

import scala.concurrent.duration.*
import cats.effect.Temporal

def retryIO[A](ioa: IO[A], maxRetries: Int, delay: FiniteDuration)(implicit T: Temporal[IO]): IO[A] = {
  ioa.handleErrorWith { error =>
    if (maxRetries > 0) {
      // Add specific checks here, e.g., only retry on 429 or network errors
      T.sleep(delay) *> retryIO(ioa, maxRetries - 1, delay * 2) // Exponential backoff
    } else {
      IO.raiseError(error) // Max retries reached
    }
  }
}

// Example usage:
// val uploadIO = fromCompletableFuture(...)
// retryIO(uploadIO, maxRetries = 3, delay = 1.second).flatMap(...)

Upload many files in parallel

Use parTraverseN from Cats Effect to upload multiple files concurrently, limiting the concurrency to avoid overwhelming resources.

import cats.syntax.all.*
import cats.effect.implicits.*

case class FileTask(localPath: String, targetKey: String)

def uploadFileAsync(task: FileTask): IO[PutObjectResponse] = {
  val putReq = PutObjectRequest.builder()
    .bucket(bucketName)
    .key(task.targetKey)
    .build() // ContentType could be inferred or set dynamically
  fromCompletableFuture {
    s3AsyncClient.putObject(putReq, RequestBody.fromFile(Paths.get(task.localPath)))
  }
}

val filesToUpload = List(
  FileTask("./a.txt", "uploads/a.txt"),
  FileTask("./b.jpg", "uploads/b.jpg"),
  FileTask("./c.pdf", "uploads/c.pdf")
)

val parallelUploadsIO: IO[List[PutObjectResponse]] =
  filesToUpload.parTraverseN(4)(uploadFileAsync) // Limit to 4 concurrent uploads

// Execute parallelUploadsIO within your IOApp or effect context
// Remember error handling for individual uploads within the list

Stream large files using fs2

For large files, avoid loading the entire content into memory. Stream the file using FS2 and the AWS SDK's AsyncRequestBody.

import fs2.io.file.{Files, Path => Fs2Path}
import fs2.interop.reactivestreams.*
import software.amazon.awssdk.core.async.AsyncRequestBody
import cats.effect.kernel.Resource

def uploadStream(localPath: String, targetKey: String, contentType: String): IO[PutObjectResponse] = {
  val fs2Path = Fs2Path(localPath)

  // Resource to ensure the file stream is closed
  Files[IO].size(fs2Path).flatMap { fileSize =>
    val fileStream: fs2.Stream[IO, Byte] = Files[IO].readAll(fs2Path, 64 * 1024) // 64KB chunks

    // Convert FS2 stream to Reactive Streams Publisher
    val publisher = fileStream.toUnicastPublisher()

    Resource.eval(publisher).use { pub =>
      val requestBody = AsyncRequestBody.fromPublisher(pub)
      val putReq = PutObjectRequest.builder()
        .bucket(bucketName)
        .key(targetKey)
        .contentType(contentType)
        // .contentLength(fileSize) // Recommended for progress tracking
        .build()

      fromCompletableFuture(s3AsyncClient.putObject(putReq, requestBody))
    }
  }
}

// Example usage:
// uploadStream("./large-video.mp4", "uploads/large-video.mp4", "video/mp4").flatMap(...)

Security best practices

  1. Secrets Management: Never hardcode credentials. Use environment variables, configuration files outside source control, or dedicated secrets management services.
  2. Least Privilege: Create R2 API tokens with the minimum permissions required (e.g., only PutObject for a specific bucket).
  3. Input Validation: Validate file types, sizes, and potentially scan user-uploaded content before uploading to R2 to prevent abuse and storage of malicious files.
  4. HTTPS: Always use HTTPS. The R2 endpoint enforces this, but ensure your client configuration doesn't disable TLS verification.
  5. Bucket Policies: Configure R2 bucket policies to restrict access further, complementing API token permissions.

Common errors and their fixes

Error Likely Cause Quick Fix
403 Forbidden Invalid credentials or insufficient permissions Verify Access Key/Secret Key, check R2 token permissions
404 Not Found Bucket name typo or non-existent bucket Double-check bucket name spelling
InvalidSignatureException Incorrect signing (clock skew, wrong region) Sync server time (NTP), ensure SDK uses a valid region for signing (us-east-1)
429 Too Many Requests Exceeded R2 API request rate limits Implement exponential back-off and retry logic
SdkClientException Network issue, DNS resolution failure, timeout Check network connectivity, increase client timeouts if needed
Timeout Error Slow network or very large file upload Increase SDK client timeouts, use streaming/multipart upload

Wrap-up

Using the AWS SDK v2 for Java provides a robust and idiomatic way to interact with Cloudflare R2 from Scala. By leveraging the SDK's features combined with libraries like Cats Effect and FS2, you can build efficient, asynchronous, and resilient file export solutions. Remember to handle errors gracefully, implement retries, and follow security best practices.

If you prefer a managed solution for file processing pipelines that include exporting to various storage providers like R2, the/our 🤖 /cloudflare/store Robot automates R2 exports within a Transloadit workflow—no extra SDK integration required on your end.