Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce new versions for FaceDetector, SelfieViewModel classes and Liveness Check Manager. #224

Merged
merged 49 commits into from
Sep 10, 2024
Merged

Introduce new versions for FaceDetector, SelfieViewModel classes and Liveness Check Manager. #224

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
4080479
implement design for selfie capture camera frame.
tobitech Jul 19, 2024
d823d16
add lottie animation, product cells for selfie enrollment and selfie …
tobitech Jul 22, 2024
edfcba5
update new selfie view model with existing implementation
tobitech Jul 24, 2024
b9c89cd
add some image utility functions, import selfie quality model and add…
tobitech Jul 31, 2024
aa8519b
Merge branch 'new-smart-selfie-capture' into selfie-quality-model
tobitech Jul 31, 2024
1374bfc
some code formatting
tobitech Jul 31, 2024
8a3159d
more code formatting
tobitech Jul 31, 2024
f5a508a
compile coreml model to generate .mlmodelc file.
tobitech Jul 31, 2024
a1e727b
automatically generate the model class file.
tobitech Jul 31, 2024
50b6c70
remove uncompiled model file.
tobitech Jul 31, 2024
dd6ca7e
disable lint from generated file.
tobitech Jul 31, 2024
7a2f325
make a separate class that will handle all the image processing and c…
tobitech Aug 6, 2024
53f5e67
add some documentation and code formatting
tobitech Aug 6, 2024
ed33072
Merge branch 'main' into selfie-quality-model
tobitech Aug 7, 2024
7b8e90f
fix issue with crop to face function.
tobitech Aug 13, 2024
cafd1bb
introduce a new version for facedetector class and new debug design f…
tobitech Aug 14, 2024
de04c27
rename preview view to camera view controller
tobitech Aug 14, 2024
a450e4a
hook up new face detector class to the new selfie view model.
tobitech Aug 15, 2024
f8d3ff7
integrate the new model that takes an image input and also display th…
tobitech Aug 19, 2024
c2ea145
Merge branch 'main' into active-liveness-checks
tobitech Aug 19, 2024
97ac1f4
Merge branch 'new-smart-selfie-capture' into active-liveness-checks
tobitech Aug 19, 2024
7d44c6b
remove lottie resources. code formatting.
tobitech Aug 20, 2024
0214a84
Add new debug views and and also use av preview layer to calculate th…
tobitech Aug 20, 2024
544a48f
Fix issue with face bounding box. implement face size and positioning…
tobitech Aug 21, 2024
45d4faa
add instructions view to the selfie capture screen.
tobitech Aug 21, 2024
73d8236
add image capture to the selfie view model.
tobitech Aug 23, 2024
5a5373c
Add a view to show captured images: selfie and liveness.
tobitech Aug 23, 2024
9db6318
add active liveness manager class and move updating directive to the …
tobitech Aug 27, 2024
9ec0d9f
Merge branch 'new-smart-selfie-capture' into active-liveness-checks
tobitech Aug 27, 2024
8fa841f
some clean up
tobitech Aug 27, 2024
126caca
remove print statements from facedetectorv2
tobitech Aug 27, 2024
35e900b
move face geometry check for active liveness into the liveness manager
tobitech Aug 28, 2024
4f194c1
refactor how current task is assigned and how directive for liveness …
tobitech Aug 28, 2024
7a24d93
update criteria for passing the liveness checks
tobitech Aug 29, 2024
253d87b
fix the length of the captured image view
tobitech Aug 29, 2024
68835c2
code formatting
tobitech Aug 29, 2024
d94d1f7
add some documentation to liveness check manager class
tobitech Aug 29, 2024
3c7a96d
more docs
tobitech Aug 29, 2024
6459475
Merge branch 'main' into active-liveness-checks
tobitech Aug 29, 2024
acc6088
run pod install
tobitech Aug 29, 2024
4efc947
folder structure changes.
tobitech Aug 29, 2024
fb0ebda
code formatting.
tobitech Aug 29, 2024
ae0ff57
Merge branch 'new-smart-selfie-capture' into active-liveness-checks
tobitech Aug 29, 2024
65a51be
import UIkit inside Imageclassifier.
tobitech Aug 29, 2024
2cfcd7f
add the ML folder group into the right target.
tobitech Sep 2, 2024
9641c0c
some layout changes.
tobitech Sep 2, 2024
ae218ff
Merge branch 'new-smart-selfie-capture' into active-liveness-checks
tobitech Sep 2, 2024
4a1b6ca
update the bundle for the generated ml model class.
tobitech Sep 6, 2024
ca464b9
move initialisation for the image classifier into the init function s…
tobitech Sep 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 59 additions & 59 deletions Example/SmileID.xcodeproj/project.pbxproj
View file
Open in desktop

Large diffs are not rendered by default.

117 changes: 0 additions & 117 deletions ML/ImageClassifier.swift
View file
Open in desktop

This file was deleted.

12 changes: 12 additions & 0 deletions SmileID.xcodeproj/project.pbxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
1F33A24780AE0C89E89EDB3A /* NavigationBar.swift in Sources */ = {isa = PBXBuildFile; fileRef = 62D2BD3354878490B62F04FB /* NavigationBar.swift */; };
1F57B39849B01BB6E26B7F3B /* DocumentCaptureInstructionsScreen.swift in Sources */ = {isa = PBXBuildFile; fileRef = 540FE72848132B16535C7281 /* DocumentCaptureInstructionsScreen.swift */; };
1F76F767069D4436C8AF6074 /* Injected.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5EB66DD1BDD5A4C66AC800B3 /* Injected.swift */; };
20991A4E2C8625EE00C0B13E /* SelfieQualityDetector.mlmodelc in Resources */ = {isa = PBXBuildFile; fileRef = 20991A4C2C8625EE00C0B13E /* SelfieQualityDetector.mlmodelc */; };
216916E16C0F6C9CBD698C92 /* ImageExtensions.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0C1C11D7A28442BD26144875 /* ImageExtensions.swift */; };
2228D942A74ACE036BCC6AC3 /* OrchestratedDocumentVerificationViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = A389AF9529CF6ADA5A11D4D6 /* OrchestratedDocumentVerificationViewModel.swift */; };
225873F4B9CFD28079C4E610 /* FaceShape.swift in Sources */ = {isa = PBXBuildFile; fileRef = 60101EA1F67FE523A8D10E2F /* FaceShape.swift */; };
Expand Down Expand Up @@ -760,6 +761,7 @@
204E25AA87E9519F31645909 /* DocumentCaptureScreen.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; name = DocumentCaptureScreen.swift; path = Sources/SmileID/Classes/DocumentVerification/View/DocumentCaptureScreen.swift; sourceTree = "<group>"; };
2057FBF6B6724BE33B6CC727 /* DocumentCaptureScreen.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; name = DocumentCaptureScreen.swift; path = Sources/SmileID/Classes/DocumentVerification/View/DocumentCaptureScreen.swift; sourceTree = "<group>"; };
208563C91ED929072F239DE9 /* NavigationHelper.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; name = NavigationHelper.swift; path = Sources/SmileID/Classes/Helpers/NavigationHelper.swift; sourceTree = "<group>"; };
20991A4C2C8625EE00C0B13E /* SelfieQualityDetector.mlmodelc */ = {isa = PBXFileReference; lastKnownFileType = wrapper; path = SelfieQualityDetector.mlmodelc; sourceTree = "<group>"; };
209BE3605849ED55515BE4BF /* OrchestratedDocumentVerificationViewModel.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; name = OrchestratedDocumentVerificationViewModel.swift; path = Sources/SmileID/Classes/DocumentVerification/Model/OrchestratedDocumentVerificationViewModel.swift; sourceTree = "<group>"; };
20BD76F142CABEAE07EF1F67 /* SmileIDService.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; name = SmileIDService.swift; path = Sources/SmileID/Classes/Networking/SmileIDService.swift; sourceTree = "<group>"; };
20CAE1873FDA57190CC2F915 /* HTTPHeader.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; name = HTTPHeader.swift; path = Sources/SmileID/Classes/Networking/HTTPHeader.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -6522,6 +6524,7 @@
1EEFC2BF2B58412300B8A934 /* Resources */ = {
isa = PBXGroup;
children = (
20991A4D2C8625EE00C0B13E /* ML */,
5829A8C42BC8494F001C1E7E /* PrivacyInfo.xcprivacy */,
1EEFC2C02B58412300B8A934 /* Media.xcassets */,
1EEFC2C12B58412300B8A934 /* Fonts */,
Expand Down Expand Up @@ -6598,6 +6601,14 @@
path = DependencyContainer;
sourceTree = "<group>";
};
20991A4D2C8625EE00C0B13E /* ML */ = {
isa = PBXGroup;
children = (
20991A4C2C8625EE00C0B13E /* SelfieQualityDetector.mlmodelc */,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this may be the issue, we need this in the SDK not in the project, when installing with cocoapods, you will have to tweak the podspec and test this across spm, carthage and cocoapods as well

);
path = ML;
sourceTree = "<group>";
};
/* End PBXGroup section */

/* Begin PBXHeadersBuildPhase section */
Expand Down Expand Up @@ -6730,6 +6741,7 @@
5829A8C52BC8494F001C1E7E /* PrivacyInfo.xcprivacy in Resources */,
1EEFC3342B58412400B8A934 /* Localizable.strings in Resources */,
1EEFC3312B58412400B8A934 /* Media.xcassets in Resources */,
20991A4E2C8625EE00C0B13E /* SelfieQualityDetector.mlmodelc in Resources */,
1EEFC3332B58412400B8A934 /* Epilogue-Bold.ttf in Resources */,
1EEFC2E92B58412300B8A934 /* ARViewController.xib in Resources */,
1EEFC3322B58412400B8A934 /* Epilogue-Medium.ttf in Resources */,
Expand Down
4 changes: 3 additions & 1 deletion Sources/SmileID/Classes/Camera/CameraManager.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ class CameraManager: NSObject, ObservableObject {
@Published private(set) var status = Status.unconfigured
private var orientation: Orientation

init(orientation: Orientation) {
static let shared: CameraManager = CameraManager(orientation: .portrait)

private init(orientation: Orientation) {
self.orientation = orientation
super.init()
sessionQueue.async {
Expand Down
16 changes: 14 additions & 2 deletions Sources/SmileID/Classes/Camera/CameraViewController.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ import UIKit
import Vision
import AVFoundation

class PreviewView: UIViewController {
class CameraViewController: UIViewController {
var faceDetector: FaceDetectorV2?

var previewLayer: AVCaptureVideoPreviewLayer?
private weak var cameraManager: CameraManager?

Expand All @@ -17,6 +19,7 @@ class PreviewView: UIViewController {

override func viewDidLoad() {
super.viewDidLoad()
faceDetector?.viewDelegate = self
configurePreviewLayer()
}

Expand All @@ -34,7 +37,16 @@ class PreviewView: UIViewController {
}
}

extension PreviewView: RectangleDetectionDelegate {
extension CameraViewController: FaceDetectorDelegate {
func convertFromMetadataToPreviewRect(rect: CGRect) -> CGRect {
guard let previewLayer = previewLayer else {
return CGRect.zero
}
return previewLayer.layerRectConverted(fromMetadataOutputRect: rect)
}
}

extension CameraViewController: RectangleDetectionDelegate {
func didDetectQuad(
quad: Quadrilateral?,
_ imageSize: CGSize,
Expand Down
2 changes: 1 addition & 1 deletion Sources/SmileID/Classes/DocumentVerification/Model/DocumentCaptureViewModel.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ class DocumentCaptureViewModel: ObservableObject {
@Published var documentImageToConfirm: Data?
@Published var captureError: Error?
@Published var isCapturing = false
@Published var cameraManager = CameraManager(orientation: .portrait)
var cameraManager = CameraManager.shared

init(knownAspectRatio: Double? = nil) {
self.knownAspectRatio = knownAspectRatio
Expand Down
0 ML/CGImage+CVPixelBuffer.swift → .../FaceDetector/CGImage+CVPixelBuffer.swift
View file
Open in desktop
File renamed without changes.
Loading
Loading