Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Active Liveness Metadata #265

Merged
merged 7 commits into from
Dec 13, 2024

Conversation

jumaallan
Copy link
Member

Story: https://app.shortcut.com/smileid/story/xxx

Summary

A few sentences/bullet points about the changes

Known Issues

Any shortcomings in your work. This may include corner cases not correctly handled or issues related
to but not within the scope of your PR. Design compromises should be discussed here if they were not
already discussed above.

Test Instructions

Concise test instructions on how to verify that your feature works as intended. This should include
changes to the development environment (if applicable) and all commands needed to run your work.

Screenshot

If applicable (e.g. UI changes), add screenshots to help explain your work.

Copy link

github-actions bot commented Dec 13, 2024

Warnings
⚠️ The source files were changed, but the tests remain unmodified. Consider updating or adding to the tests to match the source changes.

Generated by 🚫 Danger Swift against 8c97d30

@jumaallan jumaallan marked this pull request as ready for review December 13, 2024 09:06
@jumaallan jumaallan requested a review from a team as a code owner December 13, 2024 09:06
@prfectionist
Copy link

prfectionist bot commented Dec 13, 2024

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Hardcoded Version
The active liveness version is hardcoded to '1.0.0'. Consider making this configurable or derived from a central version source to avoid inconsistencies when version changes are needed.

Limited Liveness Type
The code only sets 'headPose' as the liveness type, but the enum also supports 'smile'. Consider if this is intentional or if the type should be determined dynamically based on the actual liveness check being performed.

@prfectionist
Copy link

prfectionist bot commented Dec 13, 2024

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Score
Possible issue
Use dynamic values instead of hardcoded enum cases to reflect actual runtime state

The hardcoded LivenessType.headPose might not reflect the actual liveness type being
used. Consider passing the actual liveness type as a parameter.

Sources/SmileID/Classes/SelfieCapture/EnhancedSmartSelfieViewModel.swift [476]

-localMetadata.addMetadata(Metadatum.ActiveLivenessType(livenessType: LivenessType.headPose))
+localMetadata.addMetadata(Metadatum.ActiveLivenessType(livenessType: currentLivenessType))
Suggestion importance[1-10]: 8

Why: Hardcoding LivenessType.headPose could lead to incorrect metadata if different liveness types are used at runtime. Using the actual liveness type would ensure accurate tracking and prevent potential bugs.

8
Maintainability
Avoid hardcoding version strings to make version updates more maintainable

Consider making the active liveness version a configurable value rather than
hardcoding "1.0.0", as this will need to be updated with each version change.

Sources/SmileID/Classes/Networking/Models/v2/Metadata.swift [54]

-public static let activeLivenessVersion = Metadatum(name: "active_liveness_version", value: "1.0.0")
+public static let activeLivenessVersion = Metadatum(name: "active_liveness_version", value: SmileID.activeLivenessVersion)
Suggestion importance[1-10]: 7

Why: Hardcoding version strings makes updates error-prone and requires code changes. Using a configurable value would improve maintainability and reduce the risk of outdated version information.

7
Enhancement
Remove unnecessary code by eliminating redundant decoder implementations

The ActiveLivenessType class could be simplified by removing the redundant decoder
implementation since it doesn't add any functionality beyond the superclass.

Sources/SmileID/Classes/Networking/Models/v2/Metadata.swift [60-68]

 public class ActiveLivenessType: Metadatum {
     public init(livenessType: LivenessType) {
         super.init(name: "active_liveness_type", value: livenessType.rawValue)
     }
-
-    public required init(from decoder: Decoder) throws {
-        try super.init(from: decoder)
-    }
 }
Suggestion importance[1-10]: 4

Why: While the suggestion is correct about the decoder implementation being redundant, removing it has minimal impact on functionality or performance. It's a minor code cleanup improvement.

4

Copy link

@robin-ankele robin-ankele left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look fine to me. Thanks @jumaallan

@robin-ankele
Copy link

Actually, @jumaallan can you also add the liveness type smile to the legacy SelfieViewModel?

self.directive = userNeedsToSmile ? "Instructions.Smile" : "Instructions.Capturing"
self.directive =
userNeedsToSmile
? "Instructions.Smile" : "Instructions.Capturing"
}

// TODO: Use mouth deformation as an alternate signal for non-ARKit capture

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • ⚠️ TODOs should be resolved (Use mouth deformation as an al...). (todo)

@tobitech tobitech merged commit 9806243 into new-smart-selfie-capture Dec 13, 2024
3 checks passed
@tobitech tobitech deleted the feat/add-metadata-active-liveness branch December 13, 2024 15:22
jumaallan added a commit that referenced this pull request Dec 13, 2024
* New Smart Selfie Capture UI (#202)

* implement design for selfie capture camera frame.

* add lottie animation, product cells for selfie enrollment and selfie authentication with strict mode.

* control visibility of smile emblem with showAttribution flag

* make selfie frame border themable.

* Selfie Quality Check With CoreML Model (#206)

* implement design for selfie capture camera frame.

* add lottie animation, product cells for selfie enrollment and selfie authentication with strict mode.

* update new selfie view model with existing implementation

* add some image utility functions, import selfie quality model and add selfie quality check to the image buffer from camera feed.

* some code formatting

* more code formatting

* compile coreml model to generate .mlmodelc file.

* automatically generate the model class file.

* remove uncompiled model file.

* disable lint from generated file.

* make a separate class that will handle all the image processing and classification tasks for selfie quality check.

* add some documentation and code formatting

* fix issue with crop to face function.

* Replace the selfie quality model with a new version that takes an image as input.

* remove call to check selfie quality in the v2 view model.

* disable linting for model generated file.

* code formattting.

* remove resources folder and the lottie file.

* code formatting.

* move model image cropping size info to a constant.

* run pod install

* Introduce new versions for FaceDetector, SelfieViewModel classes and Liveness Check Manager. (#224)

* implement design for selfie capture camera frame.

* add lottie animation, product cells for selfie enrollment and selfie authentication with strict mode.

* update new selfie view model with existing implementation

* add some image utility functions, import selfie quality model and add selfie quality check to the image buffer from camera feed.

* some code formatting

* more code formatting

* compile coreml model to generate .mlmodelc file.

* automatically generate the model class file.

* remove uncompiled model file.

* disable lint from generated file.

* make a separate class that will handle all the image processing and classification tasks for selfie quality check.

* add some documentation and code formatting

* fix issue with crop to face function.

* introduce a new version for facedetector class and new debug design for selfie capture screen. also refactor state management and action handling for the selfie view model v2

* rename preview view to camera view controller

* hook up new face detector class to the new selfie view model.

* integrate the new model that takes an image input and also display the outputs in the debug view in real time.

* remove lottie resources. code formatting.

* Add new debug views and and also use av preview layer to calculate the bounding box

* Fix issue with face bounding box. implement face size and positioning within the frame.

* add instructions view to the selfie capture screen.

* add image capture to the selfie view model.

* Add a view to show captured images: selfie and liveness.

* add active liveness manager class and move updating directive to the view model.

* some clean up

* remove print statements from facedetectorv2

* move face geometry check for active liveness into the liveness manager

* refactor how current task is assigned and how directive for liveness task is displayed

* update criteria for passing the liveness checks

* fix the length of the captured image view

* code formatting

* add some documentation to liveness check manager class

* more docs

* run pod install

* folder structure changes.

* code formatting.

* import UIkit inside Imageclassifier.

* add the ML folder group into the right target.

* some layout changes.

* update the bundle for the generated ml model class.

* move initialisation for the image classifier into the init function so that it's done only once

* run pod install

* run pod install.

* Liveness Capture Instructions Screen (#229)

* implement design for selfie capture camera frame.

* add lottie animation, product cells for selfie enrollment and selfie authentication with strict mode.

* update new selfie view model with existing implementation

* add some image utility functions, import selfie quality model and add selfie quality check to the image buffer from camera feed.

* some code formatting

* more code formatting

* compile coreml model to generate .mlmodelc file.

* automatically generate the model class file.

* remove uncompiled model file.

* disable lint from generated file.

* make a separate class that will handle all the image processing and classification tasks for selfie quality check.

* add some documentation and code formatting

* fix issue with crop to face function.

* introduce a new version for facedetector class and new debug design for selfie capture screen. also refactor state management and action handling for the selfie view model v2

* rename preview view to camera view controller

* hook up new face detector class to the new selfie view model.

* integrate the new model that takes an image input and also display the outputs in the debug view in real time.

* remove lottie resources. code formatting.

* Add new debug views and and also use av preview layer to calculate the bounding box

* Fix issue with face bounding box. implement face size and positioning within the frame.

* add instructions view to the selfie capture screen.

* add image capture to the selfie view model.

* Add a view to show captured images: selfie and liveness.

* add active liveness manager class and move updating directive to the view model.

* some clean up

* remove print statements from facedetectorv2

* move face geometry check for active liveness into the liveness manager

* refactor how current task is assigned and how directive for liveness task is displayed

* update criteria for passing the liveness checks

* fix the length of the captured image view

* code formatting

* add some documentation to liveness check manager class

* more docs

* run pod install

* folder structure changes.

* code formatting.

* import UIkit inside Imageclassifier.

* add some views for new selfie design

* create a folder group for lottie files in the resources folder and add a new lottie file.

* add lottie animation to liveness instructions screen.

* rename selfie capture instructions view to liveness capture

* Add liveness guides view with the 3 progress arcs

* restore lottie view

* import the liveness guide lottie file into resources.

* Remove comments

* run pod install.

* run pod install

* New Camera UI (#235)

* improve code and file structure

* add the liveness guides

* add some config values to liveness guides view.

* change lottie animation frame

* run pod install

* setup animation progress time for the liiveness guide lottie animation.

* refactor the components of liveness guides view.

* run pod install.

* Connect ViewModel to New UI and Refactoring (#237)

* improve code and file structure

* add the liveness guides

* add some config values to liveness guides view.

* change lottie animation frame

* run pod install

* setup animation progress time for the liiveness guide lottie animation.

* fix homeview fore each warnings.

* connect face bounds detection to the the indicator.

* add throttling to camera feed.

* use full screen cover to present home screen products.

* control progress arc visibility based on progress.

* add a dummy submit function.

* setup timers.

* add animation to the progress fill arcs.

* refactor the components of liveness guides view.

* setup delay timer and introduce state for the current animation that should be displayed with instructions after delay.

* update arc shape init

* add new lottie files. define a guide animation enum to hold the animation details.

* some refactoring.

* create a validator class for the face observation data.

* refactor buffer image processing and communication between face detector, face validator and selfie view model. some code formatting.

* remove some debug views. some refactoring and improvements.

* make capture instruction strings localizable

* introduce current liveness task into the face validator to set the right capture instruction.

* fix cropping for selfie quality check.

* show or hide the circular ring and the liveness guides based on whether face is within bounds.

* add processing view to view captured images.

* reset animation as user is completing liveness checks

* localise instruction. use appropriate error for face not found.

* refactor image conversion for use in brightness and selfie quality checks.

* replace ObservedObject with StateObject in HomeView to prevent multiple initialising.

* run pod install

* Box Oval Interface for Selfie Capture Screen (#250)

* improve code and file structure

* add the liveness guides

* add some config values to liveness guides view.

* change lottie animation frame

* run pod install

* setup animation progress time for the liiveness guide lottie animation.

* fix homeview fore each warnings.

* connect face bounds detection to the the indicator.

* add throttling to camera feed.

* use full screen cover to present home screen products.

* control progress arc visibility based on progress.

* add a dummy submit function.

* setup timers.

* add animation to the progress fill arcs.

* refactor the components of liveness guides view.

* setup delay timer and introduce state for the current animation that should be displayed with instructions after delay.

* update arc shape init

* add new lottie files. define a guide animation enum to hold the animation details.

* some refactoring.

* create a validator class for the face observation data.

* refactor buffer image processing and communication between face detector, face validator and selfie view model. some code formatting.

* remove some debug views. some refactoring and improvements.

* make capture instruction strings localizable

* introduce current liveness task into the face validator to set the right capture instruction.

* fix cropping for selfie quality check.

* show or hide the circular ring and the liveness guides based on whether face is within bounds.

* add processing view to view captured images.

* reset animation as user is completing liveness checks

* inject current liveness task into liveness guide to control which progress is showing. update reset delay timer on main thread. use appropriate error for face detection during cropping.

* processing screen layout. introduce a backport of stateobject.

* run pod install to import missing lottie files.

* present selfie capture flow in navigation view, programmatically navigate to processing view based on capture status. add a circular progress view with a loader image for the selfie processing view.

* refactor view appear setup and reset selfie capture state variables

* import submit method from selfie viewmodel.

* extract submit selfie functionality into a new class to manage the submission processes.

* some refactoring

* move backport and stateobject to helper folder.

* code formatting.

* restore threshold value

* replace ObservedObject with StateObject in HomeView so that it's initialised once.

* use proxy size instead of frame for window size calculation and face layout guide positioning.

* remove stateobject backport

* use a delegate to communicate selfie submission updates to selfie view model.
add the right title to processing screen.

* update processing changes on main thread.

* inject failure reason into the api call for submitting selfie

* rename selfie submission manager

* reset the threshold for timeout for liveness check.

* remove presentation mode variable.

* introduce an environment key to manage dismissing of the selfie capture flow.

* improve error handling of selfie capture.

* localise strings.

* code formatting.

* pod install

* make loader background color themeable.

* run pod install.

* fix missing files and build errors.

* redesign selfie capture screen to use box and oval for camera area and face bounding area

* add a view to preview selfie image, also add an actions view.

* redesign the the progress arcs for active liveness.

* improvements to validating face bounding box. add a frame to selfie preview image.

* remove processing view, add a view state to control visibility of different items based on capture state.

* adjust face size and position evaluation. fix layout for attribution and retry button.

* change active liveness progress colours

* code formatting.

* update faceboundmultiplier constant.

* refactor task timer.

* improve submission handling.

* Face Validator Tests (#252)

* add test for valid face.

* add more test cases for face validator class.

* refactor out common functionality for reusability in the validator tests.

* run pod install

* run pod install.

* Liveness Timeout Images (#253)

* decouple selfieviewmodel from livenesscheckmanager

* improve object references so to prevent retain cycles.

* write custom encoding function for failure reason, replace forced failure with failure reason enum. append failure reason data to multipart form request body.

* check that submission task is nil before assigning it.

* remove unnecessary comment

* feat: update changelog (#254)

* feat: update changelog

* chore: lint fix

* fix wrong version set for fingerprintjs package and dependency name causing spm not to resolve (#257)

* added autoassign to workflow (#259)

* added autoassign to workflow

* added autoassign to workflow/fix

* pod install

* add beta tag to strict mode products.
add a cancel toolbar button to all product screens.
remove cancel button from liveness instructions screen.

* use a different multiplier for checking face bounds for selfie and liveness capture
hide liveness progress if face is not valid.

* bump build number and add haptic feedback to selfie capture.

* reduce luminance threshold lowerbound

* reduce luminace threshold lower bound

* introduce a function to flip the selfie image for preview during submission.

* reduce head turn thresholds

* remove brightness check from selfie capture v2

* adjust screen brightness for selfie capture screen v2

* Chore(deps): bump rexml from 3.3.6 to 3.3.9 (#249)

Bumps [rexml](https://github.com/ruby/rexml) from 3.3.6 to 3.3.9.
- [Release notes](https://github.com/ruby/rexml/releases)
- [Changelog](https://github.com/ruby/rexml/blob/master/NEWS.md)
- [Commits](ruby/rexml@v3.3.6...v3.3.9)

---
updated-dependencies:
- dependency-name: rexml
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: JNdhlovu <[email protected]>
Co-authored-by: Tobi Omotayo <[email protected]>

* bump version number.

* replace selfie quality check with vision face quality in face detector and face validator.

* reset the submission task when there is a response or error.

* improve publisher reference for legacy selfie view model.

* use weak reference in legacy selfie view model to prevent retain cycle.

* introduce a backport of StateObject into sdk helpers

* disable idle timer when capturing selfie and enable it back when dismissed.

* use a backport of stateobject in selfie capture screen.

* bump build number.

* restore brightness check

* check orientation before analyzing camera frames.

* add a did cancel delegate method to selfie result delegate.

* Chore(deps): bump slackapi/slack-github-action (#260)

* add property to set camera name in camera manager. implement fallback for pre-ios 15 devices. add didCancel endpoint to smart selfie result delegate.

* use uniqueID for cameraname. add a delay when capturing random liveness images during timeout.

* import new typeface DMSans and define a model to manage its fonts and styles.

* add face not within frame case to face bounds state.

* remove overlay from face bounding area. introduce new version of animations.

* code formatting.

* correct instructions typo.

* change new smart selfie products name. disable lint checks for backport files. code formatting.

* rename smart selfie v2 to enhanced

* pod install.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: JNdhlovu <[email protected]>
Co-authored-by: Davina Anthony <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix UserID Refresh for New Jobs (#262)

* set the right flag for isEnroll in orchestrated enhanced selfie capture screen. update the location of selfie and livenessimages when submission fails.

* grab last selfie enrollment userid from pasteboard, use new generated user id for selfie authentication

* fix the layout issues with biometric kyc and enhanced kyc country picker.

* bump version number. remove comments.

* add encryption compliance key and value for non exempt encryption

* Device Motion Orientation (#263)

* code formatting.

* revert max pitch threshold

* correct instructions copy

* use device motion to determine device orientation and fall back to unlocked device orientation if device motion is not available.

* code formatting

* update to new lottie animation designs.

* Fix Selfie and Liveness Image Sizes, Fix Layout for Small Screen Devices (#266)

* Fixed Missing Document Type (#264)

* added missing document type in document jobs

* Update CHANGELOG.md

* add new submission status icons for success and failed states.

* remove setting initial value for user instruction.

* handle multiple faces detected and throw an error

* change order of face validator so that luminance is checked before head position.

* reduce the stroke size of the face in bounds indicator.

* remove done button and hide cancel button if submission was successful.

* use new status icons in status view.

* add a placeholder for api error 2214

* update selfie actions view parameters

* restore camera manager to have accessible initializer.
add camera session size preset for selfie capture image

* code formatting and set line limit of 2 to user instructions title.

* change capture screen layout to accommodate small screen devices.
remove cancel delegate method from selfie result.

* dismiss the selfie capture when submission is successful.

---------

Co-authored-by: Juma Allan <[email protected]>

* Adding Active Liveness Metadata (#265)

* added active liveness metadata

* added headpose metadata

* updated camera name metadata

* updated liveness type

* fix the camera info collection.

* code formatting

---------

Co-authored-by: Tobi Omotayo <[email protected]>

* update changelog.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: JNdhlovu <[email protected]>
Co-authored-by: Davina Anthony <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juma Allan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants