How to Extract|Compress zip files using Java

 import java.util.zip.ZipEntry;

import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;

extract Zip

public class ZipExtract {

    public List<String> extractZipFile(MultipartFile zipFile) {
        try (val zipInput = new ZipInputStream(zipFile.getInputStream())) {
            List<String> logFiles = new ArrayList<>();
            ZipEntry zipEntry;
            log.info("Received Zip File: {}", zipFile.getOriginalFilename());
            while ((zipEntry = zipInput.getNextEntry()) != null) {
                log.info("Extracted Log File: {}", zipEntry.getName());
                logFiles.add(IOUtils.toString(zipInput, StandardCharsets.UTF_8));
            }
            return logFiles;
        } catch (IOException e) {
            log.error("Fail to extract the zip file!{}", zipFile.getOriginalFilename(),  e);
        }
        return Collections.emptyList();
    }

}

comprese Zip

public class ZipCompress {

    public ZipOutputStream compressZip(List<List<String>> messages, OutputStream outputStream) throws IOException {
        val zos = new ZipOutputStream(outputStream);
        for (int i = 0; i < messages.size(); i++) {
            List<String> message = messages.get(i);
            val ze = new ZipEntry(message.get(0) + ".xml");
            zos.putNextEntry(ze);
            zos.write(message.get(1).getBytes(StandardCharsets.UTF_8), 0, message.get(1).length());
            zos.closeEntry();
        }
        return zos;
    }

}

Resolved: Generate XSD or WSDL jaxb classes into multiple packages via maven-jaxb2-plugin

assume your app consumes multiple XSD or WSDL files from different vendors. You might want to generate the jaxb binding classes into separately packages per vendor's schema files. in this post, I will tell you 2 ways to do so.

Use <generateDirectory>${project.build.directory}/generated-sources/xxx</generateDirectory>

add multiple executions with different configuration. for each execution, you have to specify different generateDirectory, or it won't work. the below is a sample in pom.xml.

<plugin>
    <groupId>org.jvnet.jaxb2.maven2</groupId>
    <artifactId>maven-jaxb2-plugin</artifactId>
    <version>0.14.0</version>
    <executions>
        <execution>
            <id>xxx-wsdl-generate</id>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <schemaLanguage>AUTODETECT</schemaLanguage>
                <generatePackage>com.xxx</generatePackage>
                <schemaDirectory>${project.basedir}/src/main/resources/xxx</schemaDirectory>
                <noFileHeader>true</noFileHeader>
                <schemaIncludes>
                    <include>*.wsdl</include>
                </schemaIncludes>
                <generateDirectory>${project.build.directory}/generated-sources/xxx</generateDirectory>
            </configuration>
        </execution>
        <execution>
            <id>yyy-wsdl-generate</id>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <schemaLanguage>AUTODETECT</schemaLanguage>
                <schemaDirectory>${project.basedir}/src/main/resources/yyy</schemaDirectory>
                <noFileHeader>true</noFileHeader>
                <schemaIncludes>
                    <include>*.wsdl</include>
                </schemaIncludes>
                <generateDirectory>${project.build.directory}/generated-sources/yyy</generateDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

use xjb binding files

A list of regular expression file search patterns to specify the binding files to be processed. Searching is based from the root of bindingDirectory.
If left undefined, then all *.xjb files in schemaDirectory will be processed. You can add below into <configuration></configuration> section.

    <bindingIncludes>
        <bindingInclude>*.xjb</bindingInclude>
    </bindingIncludes>

Here is a snippet of a xjb file.

<jaxb:bindings xmlns:jaxb="http://java.sun.com/xml/ns/jaxb" xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
  xmlns:xs="http://www.w3.org/2001/XMLSchema" version="2.0">
  <jaxb:bindings
    schemaLocation="xxx.xsd"
    node="/xs:schema">
    <jaxb:schemaBindings>
      <jaxb:package name="com.xxx"/>
    </jaxb:schemaBindings>
  </jaxb:bindings>
  <jaxb:bindings
    schemaLocation="yyy.xsd"
    node="/xs:schema">
    <jaxb:schemaBindings>
      <jaxb:package name="com.yyy"/>
    </jaxb:schemaBindings>
  </jaxb:bindings>
  <jaxb:bindings
    schemaLocation="zzz.wsdl"
    node="(//xs:schema)[1]">
    <jaxb:schemaBindings>
      <jaxb:package name="com.zzz.first"/>
    </jaxb:schemaBindings>
  </jaxb:bindings>
  <jaxb:bindings
    schemaLocation="zzz.wsdl"
    node="(//xs:schema)[2]">
    <jaxb:schemaBindings>
      <jaxb:package name="com.zzz.second"/>
    </jaxb:schemaBindings>
  </jaxb:bindings>
</jaxb:bindings>

common XJC Errors

you might encounter the below issues while xjc try to generate the jaxb binding classes from your schema files, to name a few in below:

Two declarations cause a collision in the ObjectFactory class

unable to honor class customization

A class/interface with the same name is already in use. Use a class customization to resolve this conflict.

In my opinion, the root cause is package conflict. Please check the error logs and it will report to you in detail which line in which schema caused the errors. Find the line in your schema and figure out the xpath for that node. Then set package to a different one should always fix the issue for your case.

You might noticed, in the above xjb sample, for zzz.wsdl, I set the package name to different values for two different nodes. in the zzz.wsdl, it has two xs:schema nodes, both of them has a same class which cause conflicts.  so I use node="(//xs:schema)[1]" to find the first xs:schema node and set its package  to zzz.first.

  <jaxb:bindings
    schemaLocation="zzz.wsdl"
    node="(//xs:schema)[1]">
    <jaxb:schemaBindings>
      <jaxb:package name="com.zzz.first"/>
    </jaxb:schemaBindings>
  </jaxb:bindings>
  <jaxb:bindings
    schemaLocation="zzz.wsdl"
    node="(//xs:schema)[2]">
    <jaxb:schemaBindings>
      <jaxb:package name="com.zzz.second"/>
    </jaxb:schemaBindings>
  </jaxb:bindings>

ghost-storage-adapter-oracle-ords, A Oracle Cloud Autonomous JSON Databases storage adapter for Ghost 5.x

npm install  ghost-storage-adapter-oracle-ords

https://www.npmjs.com/package/ghost-storage-adapter-oracle-ords

https://github.com/lengerrong/ghost-storage-adapter-oracle-ords#readme


In my last post,  how to custom storage adapters to make your self-hosted Ghost instance filesystem completely external?, I had a plan to implement a storage adapter of Ghost for Oracle Cloud Autonomous JSON Databases to leverage the always free 40Gb space provided by Oracle Cloud.

After researched Oracle ORDS REST API, I implement a orcal-ords-client lib to encapsulate the REST APIs for easy usage. Then I developed this storage adapter based on the orcal-ords-client lib.

Core Idea

The storage adapter is to save the images uploaded to Ghost into external rather than the file system where your Ghost instance deployed. I designed the below JSON document for a image:

{
  "path": "the unique path of the image, just consider the image is saved to local file system",
  "blob": "the json string of a Buffer object which is the content of the image, maximum size is 4.4M after tested",
  "type": "the mimetype of the image, such as png, jpeg and etc, used to set the content-type when the image been requested"
}

With the three interfaces provided by the orcal-ords-client lib,
 putJSONDocument(alias: string, json: JSONValue): Promise<Item>;
 deleteJSONObject(alias: string, id: string): Promise<boolean>;
 queryJSONDocument(alias: string, query: any, payload: any): Promise<Collection>;
We are able to implement the required methods to custom the storage adapter.
delete: first use queryJSONDocument to find the id 
of the image stored by path, and then delete the image via deleteJSONObject
exists: use queryJSONDocument to find the image stored by path
save: construct the JSON object and save into database via putJSONDocument
serve: use queryJSONDocument to find the image stored by path and then
pipe the Buffer content to response stream
read:
use queryJSONDocument to find the image stored by path and
return the Buffer from the blob field.

Ghost Configuration

"storage": {
  "active": "OracleImagesStorage",
  "OracleImagesStorage": {
    "oauthClients": [
        {
            "client_id": "YOUR_ORACLE_AUTONOMOUS_JSON_DATABASE_OAUTH_CLIENT_ID",
            "client_secret": "YOUR_ORACLE_AUTONOMOUS_JSON_DATABASE_OAUTH_CLIENT_SECRET",
            "schema": "YOUR_ORACLE_AUTONOMOUS_JSON_DATABASE_USER",
            "ords_url": "YOUR_ORACLE_AUTONOMOUS_JSON_DATABASE_ORDS_URL",
            "alias": "YOUR_ORACLE_AUTONOMOUS_JSON_DATABASE_COLLECTION_NAME"
        },
    ]
  }
}
You can put multiple Oracle Cloud Autonomous JSON Databases clients in the above.
You can get 40Gb free space to use per account. if you have 10 Oracle Cloud account,
You can use 400Gb free space.:)

Use this adapter in your Ghost

In my another blog(https://errong.win) hosted on free Google Computer Engine,
It is a self hosted Ghost instance. The below photo is served with path starting
with /content/images/, but you can't any images under the /content/images directory
on my GCP vm's local file system.



oracle-ords-client, a javascript lib to persistent your Oracle Cloud Autonomous JSON Databases via Oracle ORDS Database Management REST API


oracle-ords-client

oracle-ords-client is a JavaScript library that defines a simple and consistent abstraction for interacting with a Oracle Cloud Autonomous Databases via Oracle ORDS Database Management REST API.

declare class OracleORDSClient implements ORDSOAuthClient {
    config: ORDSOAuthClientConfig;
    private access_token;
    private expires_at;
    constructor(config: ORDSOAuthClientConfig);
    putJSONDocument(alias: string, json: JSONValue): Promise<Item>;
    deleteJSONObject(alias: string, id: string): Promise<boolean>;
    queryJSONDocument(alias: string, query: any, payload: any): Promise<Collection>;
    private buildUrl;
    private ensureOauthToken;
}

npm install oracle-ords-client

https://www.npmjs.com/package/oracle-ords-client

github home page

https://github.com/lengerrong/oracle-ords-client

Maximum JSONDocument size: 4.4M

Usage Sample:


References:

How To Setup OAuth Clients to Connect Your Oracle Cloud Autonomous Databases via REST

cy.request with FormData and QueryString

correct FormData snippet

please pass form: true and with the payload as body rather than use FormData

cy.request({
    url,
    method: "post",
    headers: {
      Authorization: "Basic xxx",
      Accept: "*/*",
      "Cache-Control": "no-cache",
      "Accept-Encoding": "gzip, deflate, br"
    },
    form: true,
    body: {
      "xxx":"yyy"
    }
  }).then((response) => {})

I tried create a FormData object and past it as body, but didn't work.

correct QueryString snippet

please pass qs with a JSON object rather than use URLSearchParams

cy.request({
      url,
      method: "post",
      headers: {
        Authorization: `Bearer ${access_token}`,
      },
      qs: {
        "parametername": "parametervalue"
      }
    });

I tried create a URLSearchParams  object and past it as qs, but didn't work.

cy.intercept won't work if the request not send by browser or send via cy.request

the below code will not work. the wait for the request alias will timeout with error: expect request of url but never occurred.

const url = "your api"
cy.intercept(url).as("yourapi");
cy.request(url);
cy.wait("@yourapi");

Spring JPA: Dynamic Schema|Table at runtime

named parameter bind only works on the where clause. what if you want different schema at runtime for the table part. such as

'select * from :tablename where ...'  
'update :tablename set ..."

You will find the above won't work in your @Query annotation

use EntityManager::createNativeQuery

You can get the single result, the first result, result lists or result stream.

use entityManager.createNativeQuery(sql).executeUpdate()

use executeUpdate to update or delete records in the table

Not allowed to create transaction on shared EntityManager

Caused by: java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

Please call joinTransaction before execute update sql

Resolved: SSL Connection Reset

 A java app encountered a weird SSL Connection Reset issue while upgrading to use java11. it talked to a legacy service, it works very well while running with java8. somehow 40% percent requests will failure in SSLException after java11.

  java.net.SocketException: Connection reset
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
  	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:110)
  	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1408)
  	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1314)
  	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
  	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411)

Cause

The legacy server is running on Oracle WebLogic Server with java8. the server will reset the connection sometimes due to unknown reason.

Analysis

We are using HttpClients custom the default request config.

CloseableHttpClient httpClient = HttpClients.custom().setDefaultRequestConfig(config)      .setSSLSocketFactory(sslCsf).setConnectionManager(cm).build();

I thought it might be cipher suite not match or socket timeout issue. It turns out the not migrated app also will receive the connect reset from the legacy server after enable the logs. The difference from the logs are:

java.net.SocketException: Connection reset // the existed app with java8 runtime
javax.net.ssl.SSLException: Connection reset // the new migrated app with java 11 runtime

I also observed the retry log from the existed app, but not see them in the new one.
org.apache.http.impl.execchain.RetryExec : Retrying request to {s}

Which makes sense, the existed app will retry the requests and got successfully response finally when connection reset happened. but the new app just throw the exception.

Why requested failed with SSLException not retry, but SocketException retried?

The default http request retry handler will not retry for the below 4 exceptions.

Solution

Write a custom http client retry handler and retry the request even encountered javax.net.ssl.SSLException: Connection reset.

You can implement your own HttpRequestRetryHandler class and set it as the retry handler for your http client.

I just copy the same implementation of DefaultHttpRequestRetryHanlder and removed SSLException.class

CloseableHttpClient httpClient = HttpClients.custom().setDefaultRequestConfig(config)
					.setSSLSocketFactory(sslCsf).setConnectionManager(cm)
					.setRetryHandler(new HttpRequestRetryHandler())

Debugging SSL/TLS Connections

use the following command-line option on the java command:

java -Djavax.net.debug=all

The following example shows potential logging settings in application.properties:

logging:
  level:
    org:
      apache:
        http: DEBUG

References:

https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html
https://docs.spring.io/spring-boot/docs/2.1.18.RELEASE/reference/html/boot-features-logging.html

fixed: embedded-redis: Unable to run on macOS Sonoma

Issue you might see below error while trying to run embedded-redis for your testing on your macOS after you upgrade to Sonoma. java.la...