Milestone 3: Synthetic Optimization #11

chris24sahadeo · 2019-07-25T21:31:36Z

Write an application that uses the ceres-solver optimization library to compute the camera’s intrinsic parameters, camera’s poses, and the fiducial poses. The optimization will use the following data

Input Data:

AprilTag detection data. (YAML files from milestone 1)

Variable Parameters (Each parameter will need an initial guess)

Camera Intrinsics
camera_T_world (for each image)
world_T_target (for each fiducial target)

Constants:

The size of each fiducial target.

Switching back to "camera_class_and_detections" To make changes to mileston

…ana/robot_camera_calibration into synthetic_optimization

...but I don't know why... Transform hacks required. NB: pixel points no longer casted to int in camera.yaml

and this time on purpose!!! Thanks to Vijay's, Kiran's and Sarika's input. Major refactoring required before PR.

Need to test with real data.

Need to introduce code to deal with models.

Need to code YAML output.

…obot_camera_calibration into synthetic_optimization

rviz_simulator/include/rviz_simulator/camera_calibration_optimizer.h

Need to fix camera yaml output.

Included <camera_calibration_parsers/parse.h> Still some code refactoring to be done.

Added: - Outputting of optimized data to YAML files.

Following optimization, pixel coordinates of corner points are recalculated and added to YAML output.

shivrar

First half finished. Will review from rviz_simulator/src/camera_calibration_optimizer.cpp tomorrow.

shivrar · 2019-08-13T14:35:15Z

rviz_simulator/CMakeLists.txt

@@ -165,22 +185,54 @@ include_directories(include
 ## add_executable(one_marker src/one_marker.cpp)
 ## target_link_libraries(one_marker ${catkin_LIBRARIES})

+FIND_PACKAGE(yaml-cpp REQUIRED)


Is this FIND_PACKAGE different than the calls above? If not, you should just use one version and also group these above.

I'll address all CMakeList.txt-related comments in a separate PR after milestone 3 is merged, since the CMakeList.txt file from milestone 2 is the same file in milestone 3 (which is a branch of 2) and changing the file in one branch and merging with the other may result in merge conflicts.

shivrar · 2019-08-13T14:47:00Z

rviz_simulator/CMakeLists.txt

+FIND_PACKAGE(yaml-cpp REQUIRED)
+
+
+# simulate
 add_executable(simulate 


NIT: Maybe for readability group all similar declarations together. Example:

add_executable(exe1 path_to_exe1) add_executable(exe2 path_to_exe2) ... add_library(lib1 lib1_files) add_library(lib2 lib2_files) target_link_libraries(exe1 lib1 other_addtional_libs ...) target_link_libraries(exe2 lib2 lib1 other_addtional_libs ...)

shivrar · 2019-08-13T14:47:15Z

rviz_simulator/CMakeLists.txt

@@ -216,4 +268,4 @@ target_link_libraries(simulate
 #   # myfile1
 #   # myfile2
 #   DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
-# )
+# )


Add new line here

shivrar · 2019-08-13T14:48:06Z

rviz_simulator/CMakeLists.txt

@@ -1,21 +1,37 @@
+# CERES


Should look at removing any irrelevant or stale comments.

rviz_simulator/README.md

rviz_simulator/src/camera.cpp

shivrar · 2019-08-13T19:16:58Z

rviz_simulator/src/camera.cpp

+  detections_out << YAML::Value << YAML::BeginSeq;  // detections sequence
+
+  // processing all camera_T_target transforms
+  for (int i = 0; i < this->world_T_targets_.size(); i++)


You can use the iterating options of a vector to iterate over the targets
either
https://en.cppreference.com/w/cpp/container/vector

Suggested change

for (int i = 0; i < this->world_T_targets_.size(); i++)

for (Eigen::Affine3d world_T_target : this->world_T_targets_)

or
http://www.cplusplus.com/reference/vector/vector/begin/

Suggested change

for (int i = 0; i < this->world_T_targets_.size(); i++)

for (std::vector<Eigen::Affine3d>::iterator it = this->world_T_targets_.begin() ; it != this->world_T_targets_.end(); ++it)

shivrar · 2019-08-13T19:21:48Z

rviz_simulator/src/camera.cpp

+  for (int i = 0; i < this->world_T_targets_.size(); i++)
+  {
+    // getting camera_T_target
+    Eigen::Affine3d camera_T_target = world_T_camera.inverse() * this->world_T_targets_[i];


With the iterator above this can become Eigen::Affine3d camera_T_target = world_T_camera.inverse() * (*it);
or something like this

shivrar · 2019-08-13T19:29:06Z

rviz_simulator/src/camera.cpp

+  differences.push_back(corners[1] - corners[3]);
+  differences.push_back(corners[0] - corners[2]);
+
+  for (int i = 0; i < differences.size(); i++)


similar note to iterate through the vectors from above.

shivrar

Some comments In line but looks like you have the right idea for stuff.

rviz_simulator/src/camera_calibration_optimizer.cpp

shivrar · 2019-08-14T17:23:05Z

rviz_simulator/src/camera_calibration_optimizer.cpp

+  for (YAML::const_iterator target_iterator = targets_node["targets"].begin();
+       target_iterator != targets_node["targets"].end(); ++target_iterator)
+  {
+    const YAML::Node& target_node = *target_iterator;


Nit: This declaration infers that the value in target_iterator is of const value and since we are treating this value as read_only we should just use the target_iterator alone.

Update: Just realized that this is okay to do but I think you should use the iterator instead of introducing a new variable.

I did it this way to make the code simpler/more readable, will consider changing it.

shivrar · 2019-08-14T17:30:32Z

rviz_simulator/src/camera_calibration_optimizer.cpp

+    Target target;
+
+    // targetID
+    target.targetID = target_node["targetID"].as<int>();


With the change above:

Suggested change

target.targetID = target_node["targetID"].as<int>();

target.targetID = target_iterator->operator[]("targetID").as<int>();

or

Suggested change

target.targetID = target_node["targetID"].as<int>();

target.targetID = (*target_iterator)["targetID"].as<int>();

rviz_simulator/src/camera_calibration_optimizer.cpp

…ana/robot_camera_calibration into synthetic_optimization

chris24sahadeo

Reviewed Shivan's comments

chris24sahadeo · 2019-08-15T19:27:09Z

rviz_simulator/README.md

+
+where the `detections_directory_name` is a directory of camera, targets and detection yaml files in the `rviz_simulator` package directory.  
+
+The output is written to a folder labelled "`optimized`" which is created in the `detections_directory_name` folder.


rviz_simulator/src/camera.cpp

chris24sahadeo · 2019-08-19T12:41:50Z

rviz_simulator/src/camera_calibration_optimizer.cpp

+  for (YAML::const_iterator target_iterator = targets_node["targets"].begin();
+       target_iterator != targets_node["targets"].end(); ++target_iterator)
+  {
+    const YAML::Node& target_node = *target_iterator;


I did it this way to make the code simpler/more readable, will consider changing it.

rviz_simulator/include/rviz_simulator/camera_calibration_optimizer.h

rviz_simulator/src/camera_calibration_optimizer.cpp

chris24sahadeo added 12 commits July 17, 2019 11:10

Optimization Code writen before the "Big Meeting"

e3773c7

Switching back to "camera_class_and_detections" To make changes to mileston

Uninitialized Jacobian Entry Error

142b141

switching to milestone 1 to fix target orientation

d28e362

Merge branch 'camera_class_and_detections' of https://github.com/Virt…

02670eb

…ana/robot_camera_calibration into synthetic_optimization

It works!!!

4079632

...but I don't know why... Transform hacks required. NB: pixel points no longer casted to int in camera.yaml

It works!!!

8ae175c

and this time on purpose!!! Thanks to Vijay's, Kiran's and Sarika's input. Major refactoring required before PR.

Works with simulated data.

c8097df

Need to test with real data.

ROSCpp Code Format

fecb0e0

Hardcoded to use the plumb_bob model.

bc6ae2c

Need to introduce code to deal with models.

Code refactored into a class

36c18f0

Need to code YAML output.

Code refactored into a classNeed to code YAML output.

5e0cbf7

Merge branch 'synthetic_optimization' of https://github.com/Virtana/r…

4cd8ba1

…obot_camera_calibration into synthetic_optimization

chris24sahadeo commented Jul 29, 2019

View reviewed changes

rviz_simulator/include/rviz_simulator/camera_calibration_optimizer.h Outdated Show resolved Hide resolved

chris24sahadeo added 3 commits July 30, 2019 18:15

Added targets YAML output.

b9a14f6

Need to fix camera yaml output.

All YAML output working.

6276898

Included <camera_calibration_parsers/parse.h> Still some code refactoring to be done.

REVIEWME.

2528168

Added: - Outputting of optimized data to YAML files.

chris24sahadeo requested a review from sarika93 July 31, 2019 19:51

Added reprojection of pixel point.

60cc705

Following optimization, pixel coordinates of corner points are recalculated and added to YAML output.

chris24sahadeo requested a review from shivrar August 13, 2019 14:15

shivrar reviewed Aug 13, 2019

View reviewed changes

shivrar reviewed Aug 14, 2019

View reviewed changes

shivrar changed the base branch from master to camera_class_and_detections August 16, 2019 19:14

Merge branch 'camera_class_and_detections' of https://github.com/Virt…

b63b020

…ana/robot_camera_calibration into synthetic_optimization

chris24sahadeo commented Aug 19, 2019

View reviewed changes

Changes based on Shivan's review

9d3455b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Milestone 3: Synthetic Optimization #11

Milestone 3: Synthetic Optimization #11

chris24sahadeo commented Jul 25, 2019

shivrar left a comment

shivrar Aug 13, 2019

chris24sahadeo Aug 15, 2019

shivrar Aug 13, 2019

shivrar Aug 13, 2019

shivrar Aug 13, 2019

shivrar Aug 13, 2019

shivrar Aug 13, 2019

shivrar Aug 13, 2019

shivrar left a comment

shivrar Aug 14, 2019

shivrar Aug 14, 2019

chris24sahadeo Aug 19, 2019

shivrar Aug 14, 2019

chris24sahadeo left a comment

chris24sahadeo Aug 15, 2019

chris24sahadeo Aug 19, 2019

	for (int i = 0; i < this->world_T_targets_.size(); i++)
	for (Eigen::Affine3d world_T_target : this->world_T_targets_)

	for (int i = 0; i < this->world_T_targets_.size(); i++)
	for (std::vector<Eigen::Affine3d>::iterator it = this->world_T_targets_.begin() ; it != this->world_T_targets_.end(); ++it)

	target.targetID = target_node["targetID"].as<int>();
	target.targetID = target_iterator->operator[]("targetID").as<int>();

	target.targetID = target_node["targetID"].as<int>();
	target.targetID = (*target_iterator)["targetID"].as<int>();


		where the `detections_directory_name` is a directory of camera, targets and detection yaml files in the `rviz_simulator` package directory.

		The output is written to a folder labelled "`optimized`" which is created in the `detections_directory_name` folder.

Milestone 3: Synthetic Optimization #11

Are you sure you want to change the base?

Milestone 3: Synthetic Optimization #11

Conversation

chris24sahadeo commented Jul 25, 2019

shivrar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shivrar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chris24sahadeo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment