How to use this repo to do OCR job.

[zydjohnHotmail]

Hi,
I am rather new for Florence2 language model, yesterday, I just received my PC with AMD 370hx CPU, I think this new PC is powerful enough to run some language model, so I want to use this repo to do some OCR job, I just want to scan the image file, let's say, I saved it as: "C:\Images\OCRTest.png".
But I can't find a good example C# code to do the OCR on this image.
Please show me some example code for this.
By the way, I am using Windows 11 with Visual Studio version: 17.12.3, I want to write one WinForms project target x64 CPU with .net 9.0.
Thanks,

[budgetdevv]

Hi @zydjohnHotmail, thanks for your interest. Unfortunately, I've yet to port OCR code over. Currently it only does image captioning ( GenerateCaption(), GenerateDetailedCaption() and GenerateMoreDetailedCaption() ).

I plan to eventually port this, but no estimated timeframe.

[zydjohnHotmail]

Hi,
I am trying to write one C# class library for part of your repo, including FlorenSharp & ONNX.Common. If I can make such class library, I want to see if I can write some C# code to do OCR job.
However, when I tried to create a class library for ONNX.Common, I found some tricky issues, here is your .csproj file for ONNX.Common:

<PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Nullable>enable</Nullable>
    <LangVersion>preview</LangVersion>
</PropertyGroup>

<ItemGroup>
    <PackageReference Include="Microsoft.ML.OnnxRuntime" Version="1.18.1" />
    <PackageReference Include="Microsoft.ML.OnnxRuntime.Extensions" Version="0.12.0" />
    <PackageReference Include="SixLabors.ImageSharp" Version="3.1.5" />
    <PackageReference Include="System.Numerics.Tensors" Version="9.0.0-preview.7.24405.7" />
</ItemGroup>

<ItemGroup>
    <EmbeddedResource Include="Resources\Embedded\*.*" Visible="true" />
</ItemGroup>

<!--  https://youtrack.jetbrains.com/issue/RIDER-26906 -->
<Target Name="ExcludeAnnoyingDylib" AfterTargets="ResolveAssemblyReferences">
    <ItemGroup>
        <None Update="@(None)">
            <Visible>false</Visible>
        </None>
    </ItemGroup>
</Target>

<PropertyGroup>
    <!-- Suppress warnings about using experimental feature -->
    <NoWarn>SYSLIB5001</NoWarn>
</PropertyGroup>

=> But when I tried to create a class library for this ONNX.Common, I want to use latest version of Nuget packages, so my .csproj file looks like this now:

<PropertyGroup>
	<TargetFramework>net9.0</TargetFramework>
	<Nullable>enable</Nullable>
	<LangVersion>preview</LangVersion>
	<!-- Suppress warnings about using experimental feature -->
	<NoWarn>CS0618;SYSLIB5001</NoWarn>
</PropertyGroup>

<ItemGroup>
	<PackageReference Include="Microsoft.ML.OnnxRuntime" Version="1.20.1" />
	<PackageReference Include="Microsoft.ML.OnnxRuntime.Extensions" Version="0.13.0" />
	<PackageReference Include="SixLabors.ImageSharp" Version="3.1.6" />
	<PackageReference Include="System.Numerics.Tensors" Version="9.0.0" />
</ItemGroup>

<ItemGroup>
	<EmbeddedResource Include="Resources\Embedded\*.*" Visible="true" />
</ItemGroup>

<Target Name="ExcludeAnnoyingDylib" AfterTargets="ResolveAssemblyReferences">
	<ItemGroup>
		<None Update="@(None)">
			<Visible>false</Visible>
		</None>
	</ItemGroup>
</Target>

=> The most tricky issue is, since I am using GA version for System.Numerics.Tensors; not preview version any more, since .net was released in November, so System.Numerics.Tensors went to GA version, which is good for building a class library, however, when I tried to build the class library, I found Visual Studio 2022 shows the project is skipped, not to build at all.
Since I edit the .csproj by hand to add this part: preview

CS0618;SYSLIB5001
=> But this part seems not in correct format, so VS 2022 simply ignores this .csproj file, I can't build the class library.
Any idea on how to fix this issue?
If I can fix this issue, maybe I can help to write the code to do OCR job for your repo.
Thanks,

[budgetdevv]

@zydjohnHotmail Hmmm, I have not worked on this project in awhile ( Due to my studies ), so I do not have good insight on this. But I do not think you should create a class library for the OCR job. The FlorenceSharp library should be further developed to include this functionality, as it is tied to the underlying model.

[budgetdevv]

I hope I can get back to implementing this as soon as possible, but no ETA atm

[zydjohnHotmail]

No, actually, I already have the code to do OCR via another repo, so I don't need to do this any more.