Introduction
In this tutorial we will create an example on how to manipulate data from RSS Feed and write to files periodically using Spring Integration framework. This guide uses Spring Integration using Java configuration to retrieve data from RSS feed, manipulate and write to files.
In this example we will create both gradle and maven based build configurations to build our application. We will read the feed data from URL https://spring.io/blog.atom. We will transform to return the feed title, author and URL in a single line and write to the files. The files are generated under system’s temp directory under folder spring-integration/rss-feed. The file name ends with .msg. You can open such file using Notepad or Notepad++.
Prerequisites
Eclipse 4.12, Spring Boot 2.2.3, Spring Integration 5.2.3, Java at least 8, Gradle 5.6, Maven 3.6.1
Create Project
Create a gradle or maven based project in Eclipse with the project’s name as spring-integration-rss-feed-to-file.
Update the maven based pom.xml or gradle based build.gradle build file with the corresponding content as shown below.
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.roytuts</groupId>
<artifactId>spring-integration-rss-feed-to-file</artifactId>
<version>0.0.1-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.3.RELEASE</version>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-feed</artifactId>
<version>5.2.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-file</artifactId>
<version>5.2.3.RELEASE</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>at least 8</source>
<target>at least 8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>build.gradle
buildscript {
ext {
springBootVersion = '2.2.2.RELEASE'
}
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
}
}
plugins {
id 'java-library'
id 'org.springframework.boot' version '2.2.3.RELEASE'
}
sourceCompatibility = 12
targetCompatibility = 12
repositories {
mavenCentral()
}
dependencies {
implementation("org.springframework.boot:spring-boot-starter-integration:${springBootVersion}")
implementation("org.springframework.integration:spring-integration-feed:5.2.3.RELEASE")
implementation("org.springframework.integration:spring-integration-file:5.2.3.RELEASE")
}Spring Configuration
Now we will create Spring Integration configuration class to read data from RSS feed, transform and write to file under temp directory.
Create a class called FlowConfig and annotate with @Configuration.
Add RSS feed URL in this class as follows:
@Value(value = "https://spring.io/blog.atom")
private Resource feedResource;Write to files under temp/spring-integration directory:
@Bean
public MessageHandler targetDirectory() {
FileWritingMessageHandler handler = new FileWritingMessageHandler(
new File(System.getProperty("java.io.tmpdir") + "/spring-integration/rss-feed"));
handler.setAutoCreateDirectory(true);
// handler.setCharset("UTF-8");
handler.setExpectReply(false);
return handler;
}If the directory spring-integration/rss-feed does not exist then we create the directory automatically by setting flag to true. Under this directory you will find files are generated with the content.
We don’t want reply so we set to false.
Transform or manipulate data from RSS feed:
@Bean
public AbstractPayloadTransformer<SyndEntry, String> extractLinkFromFeed() {
return new AbstractPayloadTransformer<SyndEntry, String>() {
@Override
protected String transformPayload(SyndEntry payload) {
return payload.getTitle() + " " + payload.getAuthor() + " " + payload.getLink();
}
};
}We just append the title, author and URL from RSS feed into a single string. You may have different purpose but for simplicity we have kept the simple logic. You may also need to work on other fields of RSS feed.
Now integrate all flows together to read data from RSS feed, transform and write to file:
@Bean
public IntegrationFlow feedFlow() {
return IntegrationFlows
.from(Feed.inboundAdapter(this.feedResource, "news"), e -> e.poller(p -> p.fixedDelay(5000)))
.transform(extractLinkFromFeed()).handle(targetDirectory())
// .handle(System.out::println)
.get();
}So every 5 seconds data will be read from RSS feed and written to file.
I have also commented one line handle(System.out::println) and this line is used to print or log into the standard console.
Main Class
A class with main method and @SpringBootApplication annotation is enough to deploy the application into embedded Tomcat server.
package com.roytuts.spring.integration.rss.feed.to.file;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class SpringIntegrationRSSFeedToFileApp {
public static void main(String[] args) {
SpringApplication.run(SpringIntegrationRSSFeedToFileApp.class, args);
}
}Testing the Application
Running the above main class will give you the following output:
The file names are random with content.
When you open one of the files in an editor, such as, Notepad or Notepad++, then you will see similar to below content in it:
Spring Vault 2.2.1.RELEASE and 2.1.5.RELEASE available Mark Paluch https://spring.io/blog/2020/01/17/spring-vault-2-2-1-release-and-2-1-5-release-availableThat’s all.
Source Code
Thanks for reading.