Spring JPA: Dynamic Schema|Table at runtime

named parameter bind only works on the where clause. what if you want different schema at runtime for the table part. such as

'select * from :tablename where ...'

'update :tablename set ..."

You will find the above won't work in your @Query annotation

use EntityManager::createNativeQuery

You can get the single result, the first result, result lists or result stream.

use entityManager.createNativeQuery(sql).executeUpdate()

use executeUpdate to update or delete records in the table

Not allowed to create transaction on shared EntityManager

Caused by: java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

Please call joinTransaction before execute update sql

Resolved: SSL Connection Reset

A java app encountered a weird SSL Connection Reset issue while upgrading to use java11. it talked to a legacy service, it works very well while running with java8. somehow 40% percent requests will failure in SSLException after java11.

  java.net.SocketException: Connection reset
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
  	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
  	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
  	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:110)
  	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1408)
  	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1314)
  	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
  	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411)

Cause

The legacy server is running on Oracle WebLogic Server with java8. the server will reset the connection sometimes due to unknown reason.

Analysis

We are using HttpClients custom the default request config.

CloseableHttpClient httpClient = HttpClients.custom().setDefaultRequestConfig(config)      .setSSLSocketFactory(sslCsf).setConnectionManager(cm).build();

I thought it might be cipher suite not match or socket timeout issue. It turns out the not migrated app also will receive the connect reset from the legacy server after enable the logs. The difference from the logs are:

java.net.SocketException: Connection reset // the existed app with java8 runtime
javax.net.ssl.SSLException: Connection reset // the new migrated app with java 11 runtime

I also observed the retry log from the existed app, but not see them in the new one.
org.apache.http.impl.execchain.RetryExec : Retrying request to {s}

Which makes sense, the existed app will retry the requests and got successfully response finally when connection reset happened. but the new app just throw the exception.

Why requested failed with SSLException not retry, but SocketException retried?

The default http request retry handler will not retry for the below 4 exceptions.

Solution

Write a custom http client retry handler and retry the request even encountered javax.net.ssl.SSLException: Connection reset.

You can implement your own HttpRequestRetryHandler class and set it as the retry handler for your http client.

I just copy the same implementation of DefaultHttpRequestRetryHanlder and removed SSLException.class

CloseableHttpClient httpClient = HttpClients.custom().setDefaultRequestConfig(config)
					.setSSLSocketFactory(sslCsf).setConnectionManager(cm)
					.setRetryHandler(new HttpRequestRetryHandler())

Debugging SSL/TLS Connections

use the following command-line option on the java command:

java -Djavax.net.debug=all

The following example shows potential logging settings in application.properties:

logging:
  level:
    org:
      apache:
        http: DEBUG

References:

https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html
https://docs.spring.io/spring-boot/docs/2.1.18.RELEASE/reference/html/boot-features-logging.html

2023, New Start! New AdSense Account! New Earnings!

It's been a while after I created adsense account to display ads on this blog. A few years later the payment finally reached my payout threshold, I just found the existed adsense account is bind to China. but I moved to Canada 3 years ago. I tried to change the country of it, but just found

Unfortunately it's not possible to change the country of your payment address in AdSense. If you've moved to a new country or territory, you'll need to cancel your existing AdSense account and create a new AdSense account.

So I created a new adsense account today on 2023-01-12. Though new account is still under reviewing, I believe it will be approved soon.

To reach out payout threshold in this year, I set up the below plan:

write one blog per week day!!!

write one blog per week day!!!

Try ingress more traffic to this blog

How To Setup OAuth Clients to Connect Your Oracle Cloud Autonomous Databases via REST

Oracle Cloud provides you about 40G always free Autonomous JSON Database. That's a lot for your personal learning or blog. Let's find a way to utilize the free space in your web application. This is a step by step to set up a OAuth Client to connect your Oracle Cloud Autonomous Databases.

Create a user account

ADMIN is the default user and it has the administrator permission to your database. You don't want to your application use ADMIN user to connect. It is recommended to create a separate user per web application.

As the ADMIN user, access Database Actions and create a user with the required privileges.

Remember select "UNLIMITED" as Quota on table space DATA if you are not sure the quota size of your JSON document.

Enable REST for the new created user

Switch Authorization required on for security concern

Sign out as the ADMIN user and Sign in to Database Actions as the new user that is setting up to use OAuth authentication.

Create OAuth Client

input client information

Grant Required Roles

Grant Privileges

You just created a new OAuth Client, you will get a client id and secret.

References

What is Autonomous JSON Database?

Use SODA for REST with OAuth Client Credentials

How to custom storage adapters to make your self-hosted Ghost instance filesystem completely external?

I set up my self-hosted Ghost on a free Google Compute Engine(a E2 micro GCP VM instance) several years ago.

The free disk after running a Ghost on the vm is only about 6G left, while

Ghost uses local file storage by default

In order to save the free space, I would like to seek a way to save images into external storage.

The above code section is based on this Ghost source commit.

After dive deep into Ghost open source, you can change the above /images/upload implementation and save the file to any external storages such as AWS s3.

Or you can follow this official config guide: https://ghost.org/docs/config/#creating-a-custom-storage-adapter

You can refer to this widely tested AWS s3 adapter.

My custom idea: try to use Oracle free 40G database to save images

I always like free storage. Oracle JSON database can give you 2 instances, 20G each.

In a word, Oracle give you 40G free database storage.

I am thinking custom the upload endpoint's implementation and save the image files into Oracle JSON database.

POST http://localhost:2368/ghost/api/admin/images/upload/

The response will be like below:

{"images":[{"url":"http://localhost:2368/api/admin/images/dbx/idx","ref":null}]}

dbx the Oracle json database instance(since Oracle gives 2 free instances).

idx the json document unique id saved into Oracle json database.

We need to provide a new endpoint to parse the above response url.

Since we save it in JSON format, while browser expects an image source url to return an image binary.

This is just a proposal, I will share the implementation once done.

I implemented the custom storage adapter and let it open source. See this post to find how I build up the storage adapter.

set up jest for your typescript project and how to import txt files in your unit tests

Getting Started

These instructions will get you setup to use ts-jest in your project. For more detailed documentation, please check online documentation.

	using npm	using yarn
Prerequisites	`npm i -D jest typescript`	`yarn add --dev jest typescript`
Installing	`npm i -D ts-jest @types/jest`	`yarn add --dev ts-jest @types/jest`
Creating config	`npx ts-jest config:init`	`yarn ts-jest config:init`
Running tests	`npm t` or `npx jest`	`yarn test` or `yarn jest`

to import txt file as string, via jest-raw-loader

/** @type {import('ts-jest/dist/types').InitialOptionsTsJest} */
module.exports = {
  preset: 'ts-jest',
  testEnvironment: 'node',
  "transform": {
    "\\.txt$": "jest-raw-loader"
  }
};

or follow https://nextjs.org/docs/testing if you are using next.js

https://www.npmjs.com/package/jest-raw-loader

https://github.com/kulshekhar/ts-jest

https://jestjs.io/zh-Hans/docs/getting-started

https://jestjs.io/docs/configuration#transform-objectstring-pathtotransformer--pathtotransformer-object

Import txt files as string into your next.js App

import file from './xxx.txt'

Webpack 5 has 'Asset Modules' to allow you to import assets without configuring loaders now.

Next.js use Webpack 5 internally, so we can use 'source assets' to import txt files as strings.

Two steps to let you be able to import a txt file as a string.

custom Webpack config for next.js

        config.module.rules.push({
            test: /\.txt$/i,
            type: 'asset/source'
        })

declare txt as a module for typescript

Wildcard module declarations can be used to cover these cases.

declare module '*.txt' {
  const content: string;
  export default content;
}

example

import t1 from './1.txt'
console.log(t1)

output

event - compiled successfully

离开罪之路的人蒙祝福，

他不去随从恶人的计谋，

他不会站在罪人的道路上，

也不去坐好讥笑人的座位，

他喜爱的是耶和华律法，

昼夜默诵的也是他律法。

San Song Daddy :)