Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process skipped clients #506

Open
wants to merge 30 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
b5b5dbb
added broadcast reciever and service to reprocess the client
junaidwarsivd Jan 26, 2023
21b014b
added the corrections in the PR review
junaidwarsivd Jan 31, 2023
16470ec
added UnitTest for GizEventRepository
junaidwarsivd Jan 31, 2023
443625a
added more corrections to the PR changed the job trigger to isComplet…
junaidwarsivd Feb 1, 2023
b322055
corrected the condition
junaidwarsivd Feb 1, 2023
77e7122
removed unused imports
junaidwarsivd Feb 1, 2023
c405d49
reformatted the code
junaidwarsivd Feb 2, 2023
51d9247
updated according to the PR feedback
junaidwarsivd Feb 2, 2023
0402130
changed androidTestImplemenation to testImplementation
junaidwarsivd Feb 2, 2023
d3d2cfd
codacy issues removed
junaidwarsivd Feb 2, 2023
277e50c
added android 12 compatibility
junaidwarsivd Feb 10, 2023
89fb705
Code cleanup
ekigamba Mar 14, 2023
35ca072
Log skipped clients for reprocessing
ekigamba Mar 14, 2023
fa8589f
Process skipped clients events in-order of eventDate after processing…
ekigamba Mar 14, 2023
eb00e3d
Append APK file name with version and variant
ekigamba Dec 15, 2022
e2a4098
Separate oauth client id and secret for different variants
ekigamba Dec 15, 2022
0c4106b
Comment out code in client processor
ekigamba Dec 19, 2022
2f54b4b
Fix commented out code in client processor
ekigamba Mar 14, 2023
5c5cf23
Update ANC version to fix next button in form
ekigamba Mar 16, 2023
b4bb240
Fix OPD Close event processing
ekigamba Mar 20, 2023
6bd8dfa
Mark deceased children as closed in ec_client
ekigamba Mar 20, 2023
a834abb
Update README setup instructions
ekigamba Mar 21, 2023
07c67a8
Remove unused imports
ekigamba Mar 21, 2023
4626b37
Fix PNC registration form cannot move to the next page
ekigamba Mar 24, 2023
7e93ddf
Process death and OPD close events
ekigamba Mar 24, 2023
27b7787
Fix All Clients register search to remove dead clients
ekigamba Mar 24, 2023
502694f
Code cleanup
ekigamba Mar 24, 2023
eeab005
Update versionName and versionCode to 0.4.8 and 48
ekigamba Mar 24, 2023
57aa2d3
Fix Child register search showing deceased clients
ekigamba Mar 26, 2023
af4d112
Update versionName and versionCode to 0.4.9 and 49
ekigamba Mar 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions opensrp-giz-malawi/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,7 @@
<service android:name="org.smartregister.sync.intent.ExtendedSyncIntentService" />
<service android:name="org.smartregister.sync.intent.SettingsSyncIntentService" />
<service android:name="org.smartregister.sync.intent.SyncIntentService" />
<service android:name=".service.ReProcessSyncIntentService"/>

<uses-library
android:name="org.apache.http.legacy"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package org.smartregister.giz.application;

import android.content.Intent;
import android.content.IntentFilter;
import android.content.res.Configuration;
import android.content.res.Resources;
import android.text.TextUtils;
Expand Down Expand Up @@ -62,6 +63,7 @@
import org.smartregister.giz.job.GizMalawiJobCreator;
import org.smartregister.giz.processor.GizMalawiProcessorForJava;
import org.smartregister.giz.processor.TripleResultProcessor;
import org.smartregister.giz.recievers.SyncStatusReciever;
import org.smartregister.giz.repository.ChildAlertUpdatedRepository;
import org.smartregister.giz.repository.ClientRegisterTypeRepository;
import org.smartregister.giz.repository.DailyTalliesRepository;
Expand Down Expand Up @@ -364,6 +366,8 @@ public void onCreate() {
initOfflineSchedules();

SyncStatusBroadcastReceiver.init(this);
this.registerReceiver(new SyncStatusReciever(),
new IntentFilter(SyncStatusBroadcastReceiver.ACTION_SYNC_STATUS));
LocationHelper.init(GizUtils.ALLOWED_LEVELS, GizUtils.DEFAULT_LOCATION_LEVEL);
jsonSpecHelper = new JsonSpecHelper(this);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ public Job create(@NonNull String tag) {
return new GizVaccineUpdateJob();
case ImageUploadServiceJob.TAG:
return new ImageUploadServiceJob();
case GizReProcessJob.TAG:
return new GizReProcessJob();

default:
Timber.w(GizMalawiJobCreator.class.getCanonicalName(), "%s is not declared in Job Creator", tag);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package org.smartregister.giz.job;

import android.content.Intent;

import androidx.annotation.NonNull;

import com.evernote.android.job.Job;

import org.jetbrains.annotations.NotNull;
import org.smartregister.AllConstants;
import org.smartregister.giz.service.ReProcessSyncIntentService;
import org.smartregister.job.BaseJob;

public class GizReProcessJob extends BaseJob {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename this to GIZClientReprocessJob for now


public static final String TAG = "GizReValidationJob";

@NonNull
@NotNull
@Override
protected Result onRunJob(@NonNull @NotNull Job.Params params) {
Intent intent = new Intent(getApplicationContext(), ReProcessSyncIntentService.class);
getApplicationContext().startService(intent);
return params != null && params.getExtras().getBoolean(AllConstants.INTENT_KEY.TO_RESCHEDULE, false) ? Result.RESCHEDULE : Result.SUCCESS;

}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
package org.smartregister.giz.recievers;

import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;
import android.os.Bundle;

import org.smartregister.domain.FetchStatus;
import org.smartregister.giz.job.GizReProcessJob;

import java.io.Serializable;

import timber.log.Timber;

import static org.smartregister.receiver.SyncStatusBroadcastReceiver.EXTRA_FETCH_STATUS;

public class SyncStatusReciever extends BroadcastReceiver {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename this to GizReprocessSyncStatusReceiver since this is only being used by the GizClientReprocessJob

@Override
public void onReceive(Context context, Intent intent) {
Bundle data = intent.getExtras();
if (data != null) {
Serializable fetchStatusSerializable = data.getSerializable(EXTRA_FETCH_STATUS);
if (fetchStatusSerializable instanceof FetchStatus) {
FetchStatus fetchStatus = (FetchStatus) fetchStatusSerializable;
if (fetchStatus.equals(FetchStatus.fetched)) {
GizReProcessJob.scheduleJobImmediately(GizReProcessJob.TAG);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we want to call the job once sync is complete eg sync is done fetching all the latest records and not successfully fetched one batch. I'm not sure if this is what is currently implemented

Timber.d("Broadcast data fetched");
} else if (fetchStatus.equals(FetchStatus.nothingFetched)) {
Timber.d("Broadcast data not fetched");
} else if (fetchStatus.equals(FetchStatus.noConnection)) {
Timber.d("Broadcast data fetch failed");
}

//}
}
}

System.out.println("Broadcast Recieved");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,12 @@
import net.sqlcipher.Cursor;
import net.sqlcipher.database.SQLiteDatabase;

import org.smartregister.giz.util.GizUtils;
import org.smartregister.repository.BaseRepository;

import java.util.ArrayList;
import java.util.List;

public class GizEventRepository extends BaseRepository {

public boolean hasEvent(@NonNull String baseEntityId, @NonNull String eventType) {
Expand All @@ -21,4 +25,37 @@ public boolean hasEvent(@NonNull String baseEntityId, @NonNull String eventType)
}
return hasEvent;
}

public void ReprocessClients()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄 camelCase please and let's rename this to processSkippedClients or something a bit more descriptive enough that only unprocessed clients are being processed here

{
String missingClientRegister = "SELECT DISTINCT baseEntityId, formSubmissionId AS formSubmissionId FROM event WHERE baseEntityId IN (SELECT baseEntityId FROM client WHERE baseEntityId NOT IN (SELECT DISTINCT base_entity_id FROM client_register_type)) AND eventType LIKE '%Registration%'";

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure we need to alias formSubmissionId here

String missingECClient = "SELECT DISTINCT baseEntityId, formSubmissionId AS formSubmissionId FROM event WHERE baseEntityId IN (SELECT baseEntityId FROM client WHERE baseEntityId NOT IN (SELECT DISTINCT base_entity_id FROM ec_client)) AND eventType LIKE '%Registration%'";

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check previous comment on formSubmissionId

List<String> formSubmissionIDs = new ArrayList<>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This list allows duplicates which might not be handled

addFormSubmissionIds(formSubmissionIDs, missingClientRegister);
addFormSubmissionIds(formSubmissionIDs,missingECClient);


try {
if(formSubmissionIDs.size()>0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reformat the code to fix spacing in the if statement

GizUtils.initiateEventProcessing(formSubmissionIDs);
} catch (Exception e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should be generally catching Exception here

e.printStackTrace();
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Timber to print exceptions. It sends exceptions to Crashlytics



public void addFormSubmissionIds(List<String> ids, String query)
{
SQLiteDatabase database = getReadableDatabase();
Cursor cursor = database.rawQuery(query, null);
int columnIndex = cursor.getColumnIndex("formSubmissionId");
if(cursor.getCount() > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reformat the code file to fix the spacing

cursor.moveToFirst();
do {
ids.add(cursor.getString(columnIndex));
} while (cursor.moveToNext());
}
cursor.close();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package org.smartregister.giz.service;

import android.app.IntentService;
import android.content.Intent;

import androidx.annotation.Nullable;

import org.smartregister.giz.application.GizMalawiApplication;

public class ReProcessSyncIntentService extends IntentService {

public static final String TAG = "ReValidateSyncIntentService";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have the tag similar to the name of the service


public ReProcessSyncIntentService()
{
super(TAG);
}

public ReProcessSyncIntentService(String name) {
super(name);
}

@Override
protected void onHandleIntent(@Nullable Intent intent) {
GizMalawiApplication.getInstance().gizEventRepository().ReprocessClients();
}
}