Skip to content

TRUNK-6409: Initializer module should not re-run initialization on each node #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
IamMujuziMoses wants to merge 3 commits into
base: main
Choose a base branch
from
Open

TRUNK-6409: Initializer module should not re-run initialization on each node #307

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6cc66c7
TRUNK-6409: Initializer module should not re-run initialization on ea…
IamMujuziMoses Feb 20, 2026
b916fd5
TRUNK-6409: Initializer module should not re-run initialization on ea…
IamMujuziMoses Feb 20, 2026
ff6848d
TRUNK-6409: Initializer module should not re-run initialization on ea…
IamMujuziMoses Mar 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions api-2.5/src/main/java/org/openmrs/module/initializer/api/loaders/LiquibaseLoader2_5.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,9 +88,4 @@ private int sqlCount(String sql) {
private void sqlUpdate(String sql) {
Context.getAdministrationService().executeSQL(sql, false);
}

@Override
public ConfigDirUtil getDirUtil() {
return new ConfigDirUtil(iniz.getConfigDirPath(), iniz.getChecksumsDirPath(), getDomainName(), true);
}
}
26 changes: 25 additions & 1 deletion api/src/main/java/org/openmrs/module/initializer/InitializerActivator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,14 +82,38 @@ public void started() {
log.info("OpenMRS config loading process disabled at initializer startup");
} else {
boolean throwError = PROPS_STARTUP_LOAD_FAIL_ON_ERROR.equalsIgnoreCase(startupLoadingMode);
log.info("OpenMRS config loading process started...");
boolean lockAquired = false;
String lockName = "initializer";

try {
log.info("Waiting for initializer lock...");
getInitializerService().acquireLockOrWait(lockName, 15 * 60 * 1000); // 15 minutes
lockAquired = true;

// Check if config changed
if (!getInitializerService().isConfigChanged()) {
log.info("No config changes... skipping initializer");
return;
Copy link
Copy Markdown

@rkorytkowski rkorytkowski Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be just skipping. It should be waiting until lock is released.

Copy link
Copy Markdown
Author

@IamMujuziMoses IamMujuziMoses Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I get it now, thanks for the review

Copy link
Copy Markdown

@rkorytkowski rkorytkowski Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean it should wait and then skip. The wait is so that we don't start up another instance before initialization is done.

}

// Run Initializer
log.info("OpenMRS config loading process started...");
getInitializerService().loadUnsafe(true, throwError);
Copy link
Copy Markdown

@rkorytkowski rkorytkowski Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that with your changes if only a single checksum is different, then all files are being loaded again instead of just the one that changed... It's not correct. Only that single file should run. I'll fix that issue.


// Save new checksums
getInitializerService().updateChecksums();
Copy link
Copy Markdown

@rkorytkowski rkorytkowski Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not updateChecksums all at once... It's possible that some files get imported and checksums should be updated and some fail... I'll address that as well.

log.info("OpenMRS config loading process completed.");
}
catch (Exception e) {
log.error("Initializer failed", e);
throw new ModuleException("An error occurred loading initializer configuration", e);
}
finally {
if (lockAquired) {
getInitializerService().releaseLock(lockName);
log.info("Initializer lock released");
}
}
}
}

Expand Down
2 changes: 0 additions & 2 deletions api/src/main/java/org/openmrs/module/initializer/InitializerConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,6 @@ public class InitializerConstants {
*/
public static final String DIR_NAME_CONFIG = "configuration";

public static final String DIR_NAME_CHECKSUM = "configuration_checksums";

public static final String DIR_NAME_REJECTIONS = "configuration_rejections";

/*
Expand Down
2 changes: 1 addition & 1 deletion api/src/main/java/org/openmrs/module/initializer/InitializerMessageSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ public synchronized void refreshCache() {
protected Map<String, Properties> getMessagePropertyResourcesFromFilesystem() {
List<OrderedPropertiesFile> messagePropertyFiles = new ArrayList<>();
for (String domain : getDomainsToScan()) {
ConfigDirUtil dirUtil = new ConfigDirUtil(iniz.getConfigDirPath(), iniz.getChecksumsDirPath(), domain, true);
ConfigDirUtil dirUtil = new ConfigDirUtil(iniz.getConfigDirPath(), domain);
List<File> propFiles = dirUtil.getFiles(PROPERTIES_EXTENSION);
for (File file : propFiles) {
messagePropertyFiles.add(new OrderedPropertiesFile(file));
Expand Down
216 changes: 2 additions & 214 deletions api/src/main/java/org/openmrs/module/initializer/api/ConfigDirUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,7 @@
package org.openmrs.module.initializer.api;

import java.io.File;
import java.io.FileInputStream;
import java.io.FilenameFilter;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
Expand All @@ -14,8 +10,6 @@
import java.util.Optional;
import java.util.stream.Stream;

import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.filefilter.DirectoryFileFilter;
import org.apache.commons.io.filefilter.FileFileFilter;
Expand All @@ -30,16 +24,8 @@
*/
public class ConfigDirUtil {

protected static final String NOT_COMPUTABLE_CHECKSUM = "not_computable_checksum";

protected static final String NOT_READABLE_CHECKSUM = "not_readadble_checksum";

public static final String CHECKSUM_FILE_EXT = "checksum";

protected static final Logger log = LoggerFactory.getLogger(ConfigDirUtil.class);

protected boolean skipChecksums = false;

/*
* The domain name, so the final part of configuration domain subdirectory. Eg.
* "locations" in "../configuration/locations"
Expand All @@ -52,33 +38,16 @@ public class ConfigDirUtil {
*/
protected String domainDirPath = "";

/*
* The absolute path to the configuration domain checksum subdirectory. Eg.
* "../configuration_checksums/locations"
*/
protected String domainChecksumsDirPath = "";

/**
* Instantiates a configuration directory utility class for the specified configuration and checksum
* files directories and for the specified domain.
*
* @param configDirPath The absolute path to the configuration directory.
* @param checksumsDirPath The absolute path to the checksum files directory.
* @param domain The domain name within the configuration directory.
* @param skipChecksums Set this to false to bypass the generation of checksums.
*/
public ConfigDirUtil(String configDirPath, String checksumsDirPath, String domain, boolean skipChecksums) {
public ConfigDirUtil(String configDirPath, String domain) {
this.domain = domain;
this.domainDirPath = Paths.get(configDirPath, domain).toString();
this.domainChecksumsDirPath = Paths.get(checksumsDirPath, domain).toString();
this.skipChecksums = skipChecksums;
}

/**
* @see #ConfigDirUtil(String, String, String, boolean)
*/
public ConfigDirUtil(String configDirPath, String checksumsDirPath, String domain) {
this(configDirPath, checksumsDirPath, domain, false);
}

public String getDomain() {
Expand All @@ -89,10 +58,6 @@ public String getDomainDirPath() {
return domainDirPath;
}

public String getDomainChecksumsDirPath() {
return domainChecksumsDirPath;
}

@Override
public String toString() {
return domainDirPath;
Expand All @@ -101,7 +66,7 @@ public String toString() {
/**
* Convenience method to get a FilenameFilter for files of a given extension.
*
* @param The dot-less extension to filter for, eg. "xml", "json".
* @param extension The dot-less extension to filter for, eg. "xml", "json".
* @return The FilenameFilter.
*/
public static FilenameFilter getExtensionFilenameFilter(final String extension) {
Expand All @@ -126,29 +91,6 @@ public static String getLocatedFilename(String domainDirPath, File file) {
.removeExtension(Paths.get(domainDirPath).relativize(file.toPath()).toString().replace(File.separator, "_"));
}

/**
* Returns the checksum of a configuration file if the file has been changed since the last time a
* checksum was saved.
*
* @param domainDirPath The absolute path to the domain directory, eg. "../configuration/locations"
* @param checksumsDirPath The absolute path to the checksum files directory, eg.
* "../configuration_checksums"
* @param configFile The config file.
* @return An empty string if the checksum hasn't changed, the new checksum otherwise.
*/
protected static String getChecksumIfChanged(String domainDirPath, String checksumsDirPath, File configFile) {
final String checksumFilename = getLocatedFilename(domainDirPath, configFile) + "." + CHECKSUM_FILE_EXT;
String checksum = computeChecksum(configFile);
return readChecksum(checksumsDirPath, checksumFilename).equals(checksum) ? "" : checksum;
}

/**
* @see #getChecksumIfChanged(String, String, File)
*/
public String getChecksumIfChanged(File configFile) {
return getChecksumIfChanged(domainDirPath, domainChecksumsDirPath, configFile);
}

/**
* Fetches recursively all the files in a domain directory (or subdirectory) based on their
* extension and that do not match the exclusion patterns.
Expand Down Expand Up @@ -204,158 +146,4 @@ public List<File> getFiles(String extension, List<String> wildcardExclusions) {
public List<File> getFiles(String extension) {
return getFiles(domainDirPath, extension, Collections.emptyList());
}

/**
* @param checksumsDirPath The absolute path to the checksum files directory, eg.
* "../configuration_checksums"
* @param checksumFilename The checksum file name.
* @return The checksum value of the config file that was last successfully loaded.
*/
protected static String readChecksum(String checksumsDirPath, String checksumFilename) {

String checksum = NOT_READABLE_CHECKSUM;

if (!new File(checksumsDirPath).exists()) {
return checksum;
}

final String checksumFilePath = Paths.get(checksumsDirPath, checksumFilename).toString();
try {
final File checksumFile = new File(checksumFilePath);
if (checksumFile.exists()) {
checksum = FileUtils.readFileToString(checksumFile, "UTF-8");
}
}
catch (Exception e) {
log.warn("Error reading checksum file: " + checksumFilePath, e);
}
return checksum;
}

/**
* Reads the saved checksum for the given configuration file.
*
* @param configFile The configuration file.
* @return The saved checksum hash value.
*/
public String readSavedChecksum(File configFile) {
final String checksumFilename = getLocatedFilename(domainDirPath, configFile) + "." + CHECKSUM_FILE_EXT;
return readChecksum(domainChecksumsDirPath, checksumFilename);
}

/**
* Compute the checksum of a configuration file.
*
* @param domainDirPath The absolute path to the domain directory, eg. "../configuration/locations"
* @param configFileName The config file name, eg. "locations.csv"
* @return The checksum of the file.
*/
protected static String computeChecksum(File configFile) {

String checksum = NOT_COMPUTABLE_CHECKSUM;

if (configFile.exists()) {
try {
// checksum = Long.toHexString( FileUtils.checksumCRC32(file) );
FileInputStream fis = new FileInputStream(configFile);
checksum = DigestUtils.md5Hex(fis);
fis.close();
}
catch (Exception e) {
log.warn("Error computing checksum of config. file: " + configFile.getName(), e);
}
}
return checksum;
}

/**
* Writes the the checksum of a configuration file into the corresponding checksum file.
*
* @param domainChecksumsDirPath The absolute path to the checksum files directory for a domain, eg.
* "../configuration_checksums/locations"
* @param checksumFilename The checksum file name inside the domain checksums files directory.
* @param checksum The checksum hash of the configuration file.
*/
protected static void writeChecksum(String domainChecksumsDirPath, String checksumFilename, String checksum) {

deleteChecksumFile(domainChecksumsDirPath, checksumFilename);

if (NOT_COMPUTABLE_CHECKSUM.equals(checksum)) {
return;
}

final String checksumFilePath = Paths.get(domainChecksumsDirPath, checksumFilename).toString();
try {
FileUtils.writeStringToFile(new File(checksumFilePath), checksum, "UTF-8");
}
catch (Exception e) {
log.error("Error writing hash ('" + checksum + "') of configuration file at: " + checksumFilePath, e);
}
}

/**
* @see #writeChecksum(String, String, String)
*/
public void writeChecksum(File configFile, String checksum) {
final String checksumFilename = getLocatedFilename(domainDirPath, configFile) + "." + CHECKSUM_FILE_EXT;
if (!skipChecksums) {
writeChecksum(domainChecksumsDirPath, checksumFilename, checksum);
}
}

/**
* Removes the specified checksum file in the specified checksums directory.
*
* @param domainChecksumsDirPath The absolute path to the checksum files directory for a domain, eg.
* "../configuration_checksums/locations"
* @param checksumFilename The checksum file name inside the domain checksums files directory.
*/
protected static void deleteChecksumFile(String domainChecksumsDirPath, String checksumFilename) {

final Path checksumFilePath = Paths.get(domainChecksumsDirPath, checksumFilename);
try {
Files.deleteIfExists(checksumFilePath);
}
catch (IOException e) {
log.warn("Error deleting the checksum file at: " + checksumFilePath.toString(), e);
}
}

/**
* @see #deleteChecksumFile(String, String)
*/
public void deleteChecksumFile(String checksumFilename) {
deleteChecksumFile(domainChecksumsDirPath, checksumFilename);
}

/**
* Deletes all checksum files for the domain covered by this instance of ConfigDirUtil.
*/
public void deleteChecksums() {
deleteFilesByExtension(domainChecksumsDirPath, CHECKSUM_FILE_EXT);
}

/**
* Recursively delete all files of a given extension inside the specified folder.
*
* @param path The absolute path to the folder.
* @param extension The extension filter for, eg. "xml".
*/
public static void deleteFilesByExtension(String path, String extension) {

Stream.of(Optional.ofNullable(new File(path).listFiles(getExtensionFilenameFilter(extension))).orElse(new File[0]))
.forEach(file -> {
try {
Files.deleteIfExists(file.toPath());
}
catch (IOException e) {
log.error("The following file could not be deleted: " + file.getPath(), e);
}
});

Stream.of(Optional.ofNullable(new File(path).list(DirectoryFileFilter.INSTANCE)).orElse(new String[0]))
.forEach(dirName -> {
deleteFilesByExtension(Paths.get(path, dirName).toString(), extension);
});
}
}
Loading