chore: Support "offline" source builds of the `feature-server` image by tchughesiv · Pull Request #5193 · feast-dev/feast

tchughesiv · 2025-03-27T15:18:11Z

What this PR does / why we need it:

We should be able to build the feature-server image entirely from source, dependencies and all. This PR almost entirely accomplishes that. With this PR we also add a new limited pip dependency group, among others, which we'll use going forward in our feature-server image builds and for operator e2e tests.

In order to accomplish the offline builds from source w/ pip, we use a tool called cachi2, which pre-fetches the sdists/archives and enables us to mount the fetched sources into an offline build container.

From a linux-x64 machine, the following command will complete an offline build from source -

$ source ./infra/scripts/offline-build.sh

The following pip modules are not yet built from source, many due to maturin related source build issues -

# we'll need to pip install anything that tries to build/use maturin from source
############
# feast deps
pydantic_core==2.27.2
watchfiles==1.0.4
rpds-py==0.23.1
# snowflake dep
cryptography==44.0.2
# aws dep
aiohttp==3.11.14
# psycopg dep
psycopg-binary==3.2.5
# milvus dep
milvus-lite==2.4.12
############

Misc

Completed build from source available here - quay.io/tchughesiv/feature-server:0.47.0-sdist

Signed-off-by: Tommy Hughes <tohughes@redhat.com>

tchughesiv · 2025-04-02T14:03:09Z

@ntkathole i've made the requested changes and am now running another build to ensure nothing breaks. thanks

ntkathole · 2025-04-02T14:09:47Z

@ntkathole i've made the requested changes and am now running another build to ensure nothing breaks. thanks

a small comment, otherwise, Looks good to me !

tchughesiv · 2025-04-02T17:18:03Z

@ntkathole the build was successful with your requested changes

ntkathole

Looks good to me!

franciscojavierarceo

lgtm

franciscojavierarceo · 2025-03-28T10:24:52Z

+        os: [ ubuntu-latest ]
    env:
-      PYTHON: 3.11
+      OS: ${{ matrix.os }}


Why this change?

because i've enabled this job to run against multiple python versions

…east-dev#5193) * support offline source builds for feature-server image Signed-off-by: Tommy Hughes <tohughes@redhat.com> * offline build review changes Signed-off-by: Tommy Hughes <tohughes@redhat.com> --------- Signed-off-by: Tommy Hughes <tohughes@redhat.com> Signed-off-by: Jacob Weinhold <29459386+j-wine@users.noreply.github.com>

The `azure` extra pinned `pymssql<2.3.3`, which resolves to 2.3.2. That release ships no macOS arm64 wheel, so `make install-python-dependencies-dev` fails building pymssql from source on Apple Silicon (feast-dev#5636). The `<2.3.3` cap was added in feast-dev#5193 because pymssql 2.3.3 dropped its Linux wheels, which the offline feature-server image build relies on. pymssql 2.3.6+ restored the Linux wheels and added macOS arm64 wheels, so `>=2.3.6` fixes Apple Silicon dev installs without regressing the offline build. Regenerated the py3.10/3.11/3.12 CI requirement locks (pymssql 2.3.2 -> 2.3.13) and the pixi lock. pymssql has no runtime dependencies, so no other packages change. Fixes feast-dev#5636

The `azure` extra pinned `pymssql<2.3.3`, which resolves to 2.3.2. That release ships no macOS arm64 wheel, so `make install-python-dependencies-dev` fails building pymssql from source on Apple Silicon (#5636). The `<2.3.3` cap was added in #5193 because pymssql 2.3.3 dropped its Linux wheels, which the offline feature-server image build relies on. pymssql 2.3.6+ restored the Linux wheels and added macOS arm64 wheels, so `>=2.3.6` fixes Apple Silicon dev installs without regressing the offline build. Regenerated the py3.10/3.11/3.12 CI requirement locks (pymssql 2.3.2 -> 2.3.13) and the pixi lock. pymssql has no runtime dependencies, so no other packages change. Fixes #5636

The `azure` extra pinned `pymssql<2.3.3`, which resolves to 2.3.2. That release ships no macOS arm64 wheel, so `make install-python-dependencies-dev` fails building pymssql from source on Apple Silicon (feast-dev#5636). The `<2.3.3` cap was added in feast-dev#5193 because pymssql 2.3.3 dropped its Linux wheels, which the offline feature-server image build relies on. pymssql 2.3.6+ restored the Linux wheels and added macOS arm64 wheels, so `>=2.3.6` fixes Apple Silicon dev installs without regressing the offline build. Regenerated the py3.10/3.11/3.12 CI requirement locks (pymssql 2.3.2 -> 2.3.13) and the pixi lock. pymssql has no runtime dependencies, so no other packages change. Fixes feast-dev#5636

tchughesiv added the ok-to-test label Mar 27, 2025

tchughesiv changed the title ~~chore: Support building feature-server images from source~~ chore: Support for building feature-server images from source Mar 27, 2025

tchughesiv changed the title ~~chore: Support for building feature-server images from source~~ chore: Support for building the feature-server image from source Mar 27, 2025

tchughesiv force-pushed the noBinary branch 3 times, most recently from 504eed8 to 005db0f Compare March 27, 2025 19:49

tchughesiv marked this pull request as ready for review March 27, 2025 20:24

tchughesiv requested a review from a team as a code owner March 27, 2025 20:24

tchughesiv changed the title ~~chore: Support for building the feature-server image from source~~ chore: Support offline source builds of the feature-server image Mar 27, 2025

tchughesiv changed the title ~~chore: Support offline source builds of the feature-server image~~ chore: Support "offline" source builds of the feature-server image Mar 27, 2025

tchughesiv marked this pull request as draft March 27, 2025 20:31

tchughesiv force-pushed the noBinary branch 3 times, most recently from 4355d95 to f4ccc80 Compare March 27, 2025 22:41

tchughesiv marked this pull request as ready for review March 28, 2025 01:41

tchughesiv requested review from franciscojavierarceo and redhatHameed March 28, 2025 01:42

tchughesiv marked this pull request as draft March 28, 2025 16:14

tchughesiv force-pushed the noBinary branch from b7d51e9 to 250e8ec Compare March 28, 2025 17:05

tchughesiv marked this pull request as ready for review March 28, 2025 20:16

tchughesiv force-pushed the noBinary branch from 250e8ec to 6f6b647 Compare March 30, 2025 14:24

jyejare reviewed Apr 1, 2025

View reviewed changes

Comment thread Makefile Outdated

tchughesiv force-pushed the noBinary branch 5 times, most recently from a2a8f41 to d362522 Compare April 1, 2025 15:56

support offline source builds for feature-server image

fdb664e

Signed-off-by: Tommy Hughes <tohughes@redhat.com>

tchughesiv force-pushed the noBinary branch from d362522 to fdb664e Compare April 1, 2025 16:00

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread sdk/python/feast/infra/feature_servers/multicloud/offline/Dockerfile.builder.arrow Outdated

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread sdk/python/feast/infra/feature_servers/multicloud/offline/Dockerfile.builder.arrow Outdated

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread infra/scripts/offline-build.sh Outdated

offline build review changes

3e33393

Signed-off-by: Tommy Hughes <tohughes@redhat.com>

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread sdk/python/feast/infra/feature_servers/multicloud/Dockerfile.dev

ntkathole approved these changes Apr 2, 2025

View reviewed changes

franciscojavierarceo approved these changes Apr 3, 2025

View reviewed changes

franciscojavierarceo enabled auto-merge (squash) April 3, 2025 12:37

tchughesiv removed the request for review from redhatHameed April 3, 2025 12:40

tchughesiv disabled auto-merge April 3, 2025 12:42

tchughesiv enabled auto-merge (squash) April 3, 2025 12:43

franciscojavierarceo disabled auto-merge April 3, 2025 12:51

franciscojavierarceo merged commit cc1fcad into feast-dev:master Apr 3, 2025

This was referenced Apr 4, 2025

ci: Revert matrix changes to linter GH Action #5218

Merged

fix: Resolve no pq wrapper import issue #5240

Merged

tchughesiv mentioned this pull request Apr 15, 2025

ci: Maturin-related builds from source #5270

Merged

This was referenced May 1, 2025

chore: Offline source image builds, use hermeto generic fetcher for arrow deps #5318

Merged

chore: Add rpms & milvus-lite / ibis binaries to hermeto fetcher for offline builds #5333

Merged

adarshsm mentioned this pull request Jul 30, 2026

fix: Bump pymssql to >=2.3.6 for macOS arm64 wheel support #6668

Merged

4 tasks

Navigation Menu

Uh oh!

chore: Support "offline" source builds of the `feature-server` image - #5193

chore: Support "offline" source builds of the `feature-server` image#5193
franciscojavierarceo merged 2 commits into
feast-dev:masterfrom
tchughesiv:noBinary

tchughesiv commented Mar 27, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

Uh oh!

ntkathole commented Apr 2, 2025

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

ntkathole left a comment

Uh oh!

franciscojavierarceo left a comment

Uh oh!

franciscojavierarceo Mar 28, 2025

Uh oh!

tchughesiv Apr 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

tchughesiv commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Misc

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

Uh oh!

ntkathole commented Apr 2, 2025

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

ntkathole left a comment

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

tchughesiv Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tchughesiv commented Mar 27, 2025 •

edited

Loading