Table of Contents
Introduction
Here in this tutorial I am going to show you how to download large file using Spring REST API. Generally when you download a file, your file is stored into file system or you load it into memory as a byte array. This is not a problem when you deal with a small file but when you download a large file (MB or GB) then it may lead to out of memory error.
I will show you here how to download the large file using RestTemplate
as well as WebClient
from Spring 5 onward. RestTemplate
is synchronous HTTP client, so it’s a blocking HTTP client. WebClient
is asynchronous and non-blocking HTTP client.
By comparing to the RestTemplate, the WebClient is:
- non-blocking, reactive, and higher concurrency with less hardware resources
- provides a functional API that takes advantage of Java 8 lambdas
- supports both synchronous and asynchronous scenarios
- supports streaming up or down from a server
Prerequisites
Java at least 8, Spring Boot Web/WebFlux 2.5.2
Download Large File
Now let’s see how I can download large file. I have two URLs from where I can download 100MB and 10GB files. 100MB file can be downloaded from https://speed.hetzner.de/100MB.bin and 10GB file can be downloaded from https://speed.hetzner.de/10GB.bin.
Using WebClient
WebClient
is non-blocking client. The file will be downloaded from the URL which is passed to the input.getDownloadUrl()
. I am getting the user input of the URL from where the file will be downloaded and the location where the file will be saved after download.
@PostMapping("/download/large/file")
public ResponseEntity<String> downloadFile(@RequestBody Input input) throws IOException {
String filename = input.getDownloadUrl().substring(input.getDownloadUrl().lastIndexOf("/") + 1);
Path path = Paths
.get(input.getDownloadPath() == null ? "/" + filename : input.getDownloadPath() + "/" + filename);
WebClient webClient = WebClient.builder().baseUrl(input.getDownloadUrl()).build();
// Get file data
Flux<DataBuffer> dataBufferFlux = webClient.get().accept(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL)
.retrieve().bodyToFlux(DataBuffer.class);
// Streams the dataBufferFlux from response instead of loading it all in memory
DataBufferUtils.write(dataBufferFlux, path, StandardOpenOption.CREATE).block();
return new ResponseEntity<String>("File Downloaded Successfully", HttpStatus.OK);
}@PostMapping("/download/large/file")
public ResponseEntity<String> downloadFile(@RequestBody Input input) throws IOException {
String filename = input.getDownloadUrl().substring(input.getDownloadUrl().lastIndexOf("/") + 1);
Path path = Paths
.get(input.getDownloadPath() == null ? "/" + filename : input.getDownloadPath() + "/" + filename);
WebClient webClient = WebClient.builder().baseUrl(input.getDownloadUrl()).build();
// Get file data
Flux<DataBuffer> dataBufferFlux = webClient.get().accept(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL)
.retrieve().bodyToFlux(DataBuffer.class);
// Streams the stream from response instead of loading it all in memory
DataBufferUtils.write(dataBufferFlux, path, StandardOpenOption.CREATE).block();
return new ResponseEntity<String>("File Downloaded Successfully", HttpStatus.OK);
}
Using RestTemplate
The getForObject()
and getForEntity()
methods of RestTemplate
load the entire response in memory. This is not suitable for downloading large files since it can cause out of memory exceptions. This example shows how to stream the response of a GET request.
The file will be downloaded from the URL which is passed to the input.getDownloadUrl()
. I am getting the user input of the URL from where the file will be downloaded and the location where the file will be saved after download.
@PostMapping("/download/large/file")
public ResponseEntity<String> downloadFile(@RequestBody Input input) {
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<Void> responseExtractor = response -> {
String filename = input.getDownloadUrl().substring(input.getDownloadUrl().lastIndexOf("/") + 1);
System.out.println("filename: " + filename);
Path path = Paths
.get(input.getDownloadPath() == null ? "/" + filename : input.getDownloadPath() + "/" + filename);
Files.copy(response.getBody(), path);
return null;
};
// URL - https://speed.hetzner.de/100MB.bin, https://speed.hetzner.de/10GB.bin
restTemplate.execute(URI.create(input.getDownloadUrl()), HttpMethod.GET, requestCallback, responseExtractor);
return new ResponseEntity<String>("File Downloaded Successfully", HttpStatus.OK);
}
Note that simply returning the InputStream
from the extractor, because by the time the execute method returns, the underlying connection and stream are already closed.
Another point is RestTemplate
is going to be deprecated and removed from future version of Spring.
The response from the REST API will be sent once your file gets downloaded. If you want to make the response asynchronous from your REST API then you can use Async or DefferedResult.
The whole source code can be downloaded later from the section Source Code.
Testing the Application
I am using Postman tool to test the file download application. make sure your Spring Boot application is up and running.

Look at the download URL and download path in the above image. So your file download URL could be anything from where you want to download the file. Download path is the path where you want to save your downloaded file.
If you continuously refresh the download path location then you will see that the file size increases periodically instead of downloading all the bytes at once.
how can I use completablefuture to get BLOB from DB asynchronously?