Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SYCL] Fix windows build and inference #8003

Merged
merged 11 commits into from
Jun 20, 2024
Merged

[SYCL] Fix windows build and inference #8003

merged 11 commits into from
Jun 20, 2024

Conversation

luoyu-intel
Copy link
Contributor

@luoyu-intel luoyu-intel commented Jun 19, 2024 •
edited
Loading

Changes:

  1. Fix SYCL exception when compiled with win-debug, caused by linking release sycl library.
  2. Add sycl debug and release presets
  3. Fix crash on Windows
  4. Remove sycl::context, sycl::queue is enough

A770 on Windows:

Building a website can be done in 10 simple steps:
 Einzelnes
Step 1: Define the purpose of your website. What is the main reason you want to build a website? Is it to sell products, provide information, or connect with customers? Knowing the purpose of your website will help you determine the design and content.
Step 2: Choose a domain name. Your domain name is the web address people will use to access your website. Choose a name that is easy to remember, relevant to your business, and available.
Step 3: Select a web hosting service. Your web hosting service will store your website's files and make them accessible to

llama_print_timings:        load time =   27234.27 ms
llama_print_timings:      sample time =       4.80 ms /   128 runs   (    0.04 ms per token, 26638.92 tokens per second)
llama_print_timings: prompt eval time =   24166.25 ms /    14 tokens ( 1726.16 ms per token,     0.58 tokens per second)
llama_print_timings:        eval time =    3548.60 ms /   127 runs   (   27.94 ms per token,    35.79 tokens per second)
llama_print_timings:       total time =   27780.29 ms /   141 tokens

@github-actions github-actions bot added build Compilation issues ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Jun 19, 2024
@airMeng airMeng requested a review from ggerganov June 19, 2024 07:58
airMeng
airMeng reviewed Jun 19, 2024
View reviewed changes
ggml.h Show resolved Hide resolved
@github-actions github-actions bot added the devops improvements to build systems and github actions label Jun 19, 2024
@mofosyne mofosyne added the Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix label Jun 19, 2024
@NeoZhangJianyu
Copy link
Collaborator

  1. Have the DPCT helper functions been verified on Linux?

  2. What's the impact of performance with this PR?

  3. The Ninja is used to build windows.
    Could you update the windows chapters in README-sycl.md?
    a. add Ninja installation.
    b. rm mingw-w64 make installation.
    c. update how to build release/debug.

@luoyu-intel
Copy link
Contributor Author

Hi @NeoZhangJianyu

  1. Have the DPCT helper functions been verified on Linux?

Yes, all UTs are passed. The model inference is correct.

  1. What's the impact of performance with this PR?

No performance change. A770 on windows(this PR) has the same performance as A770 on Linux(master)

  1. The Ninja is used to build windows.
    Could you update the windows chapters in README-sycl.md?
    a. add Ninja installation.
    b. rm mingw-w64 make installation.
    c. update how to build release/debug.

Yes, I will update the README. Visual Studio integrates Ninja as default. There are no extra installation steps for it.

@airMeng
Copy link
Collaborator

airMeng commented Jun 20, 2024

@ggerganov does CI often encounter some issues recently?

@luoyu-intel
Copy link
Contributor Author

@NeoZhangJianyu README has been updated.

NeoZhangJianyu
NeoZhangJianyu approved these changes Jun 20, 2024
View reviewed changes
Copy link
Collaborator

@NeoZhangJianyu NeoZhangJianyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good to enhance for Windows!

@airMeng airMeng merged commit de391e4 into ggerganov:master Jun 20, 2024
65 checks passed
@ggerganov ggerganov mentioned this pull request Jun 20, 2024
Merged
27 tasks
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jun 30, 2024
@luoyu-intel @NeoZhangJianyu
[SYCL] Fix windows build and inference (ggerganov#8003)
e33ffb2 
* add sycl preset

* fix debug link error. fix windows crash

* update README
@luoyu-intel luoyu-intel deleted the fix_win branch July 3, 2024 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NeoZhangJianyu NeoZhangJianyu NeoZhangJianyu approved these changes

@airMeng airMeng airMeng approved these changes

@ggerganov ggerganov Awaiting requested review from ggerganov

Assignees
No one assigned
Labels
build Compilation issues devops improvements to build systems and github actions examples ggml changes relating to the ggml tensor library for machine learning Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@luoyu-intel @NeoZhangJianyu @airMeng @mofosyne