Skip to main content

mediapipe_core 0.3.0 copy "mediapipe_core: ^0.3.0" to clipboard
mediapipe_core: ^0.3.0 copied to clipboard

Shared layer for the MediaPipe task packages: Google's engine on each platform, verified downloads, an on-demand model store and the browser runtime.

MediaPipe Tasks for Flutter #

Tested with Flutter 3.50.0 Web CI Apache-2.0 license

Google's MediaPipe Tasks for Dart and Flutter on Android, iOS, macOS, Linux, Windows and the web: detect faces and objects, track face, hand and pose landmarks, recognize gestures, classify and embed images, text and audio, segment images, and more. Every task runs Google's official MediaPipe runtime for its platform, with Google's pinned models bundled at build time.

Try the live gallery

Which package do I need? #

Package Use it for Install Guide
mediapipe_vision Face and object detection; face, hand, pose and holistic landmarks; gestures; image classification, embeddings and segmentation flutter pub add mediapipe_vision Vision
mediapipe_text Text classification, embeddings (EmbeddingGemma included) and language detection on every platform; proofreading and summarization on every platform except the web, where they are not yet available flutter pub add mediapipe_text Text
mediapipe_audio Audio classification, speech to text and text to speech, on every platform flutter pub add mediapipe_audio Audio
mediapipe_decision Yes-or-no, choice and score questions about a text (Decision Maker), on every platform flutter pub add mediapipe_decision Decision
mediapipe_retrieval Text, images and audio in one embedding space (Universal Embedder) and an on-device vector index over them (Semantic Retriever), on every platform flutter pub add mediapipe_retrieval Retrieval

Add only the families you use: each one bundles only Google's MediaPipe library for that family. Every family brings mediapipe_core, which holds the shared download code, model store and browser runtime settings (core guide), and exports its types, so you do not add or import core yourself unless your own code imports package:mediapipe_core/....

Quick start #

The packages need Flutter 3.44 or newer; the latest stable release is recommended. iOS and macOS builds with Xcode 27 need Flutter 3.47 or later. Add a family from pub.dev, from your app's root:

flutter pub add mediapipe_vision

This adds mediapipe_vision: ^0.3.0 to your app's pubspec.yaml. List the models your app uses in the same pubspec, and declare the folder they are bundled in:

flutter:
  assets:
    - assets/mediapipe/

hooks:
  user_defines:
    mediapipe_vision:
      models: [face_landmarker]

Then bundle them from the app's root, and again whenever the list changes:

dart run mediapipe_core:bundle_models

It downloads each listed model once, checks it against its pinned SHA-256 and writes it into assets/mediapipe/, so the model ships inside your app: nothing is downloaded at run time, and the app works offline from its first launch. In CI, dart run mediapipe_core:bundle_models --check fails when the folder does not match the list. Each family's XxxModels.byName lists the names it accepts.

Now create a task with the bundled model:

import 'dart:typed_data';

import 'package:mediapipe_vision/mediapipe_vision.dart';

Future<int> countFaces(Uint8List rgba, int width, int height) async {
  final landmarker = await FaceLandmarker.create(
    FaceLandmarkerOptions(model: VisionModels.faceLandmarker),
  );
  try {
    final result = await landmarker.detect(
      VisionImage.fromPixels(
        pixels: rgba,
        width: width,
        height: height,
        format: VisionPixelFormat.rgba,
      ),
    );
    return result.faceLandmarks.length;
  } finally {
    await landmarker.dispose();
  }
}

A model that is not bundled makes create throw a RuntimeUnavailableException whose fix names the pubspec entry to add. To download models at run time instead, set ModelStore.allowDownloads = true before creating tasks; Android then needs the INTERNET permission, and a sandboxed macOS app the network client entitlement.

On macOS, build the app for Apple Silicon only (two lines in its AppInfo.xcconfig; without them, release builds fail). See platform setup. Your own models work too: pass modelPath or modelBytes instead of model.

Where it runs #

Family Android iOS macOS arm64 Linux x64 Windows x64 Web
Vision ✓ ✓ ✓ ✓ ✓ ✓
Text (classify, embed, detect language) ✓ ✓ ✓ ✓ ✓ ✓
Text (EmbeddingGemma) ✓ ✓ ✓ ✓ ✓ ✓
Text (proofread, summarize) ✓ ✓ ✓ ✓ ✓
Audio ✓ ✓ ✓ ✓ ✓ ✓
Audio (transcribe, speak) ✓ ✓ ✓ ✓ ✓ ✓
Decision ✓ ✓ ✓ ✓ ✓ ✓
Retrieval ✓ ✓ ✓ ✓ ✓ ✓

Every text task runs on every platform listed except Text Proofreader and Text Summarizer, which are not yet available on the web, matching the MediaPipe Tasks Web SDK.

CPU works everywhere a family is listed, except that browsers run Decision and Retrieval on WebGPU alone, which needs a hardware adapter. GPU (Metal, OpenGL ES, WebGL, WebGPU or Android GPU) depends on the task; Macs run the vision tasks on the CPU for now, until Google confirms its Metal results since 1.1.1. On iPhones, Metal is offered for nine vision tasks and Audio Transcriber, tested on an iPhone 15 Pro (iOS 27) with Google's 1.2.2 library; iPads were not tested. Every cell is tested in CI on hosted runners, the Android emulator, the iOS Simulator and Chromium, Firefox and WebKit. The vision status table lists each task, delegate, known upstream limit and physical-device result. Ask at run time with the queryXxxCapabilities() functions: they report the supported delegates and why any other is unavailable.

More #

Origin and license #

Source is licensed under Apache-2.0, with upstream notices retained.

9
likes
160
points
1.06k
downloads

Documentation

API reference

Publisher

verified publishergoogle.dev

Weekly Downloads

Shared layer for the MediaPipe task packages: Google's engine on each platform, verified downloads, an on-demand model store and the browser runtime.

Repository (GitHub)
View/report issues
Contributing

Topics

#mediapipe #machine-learning #ffi

License

BSD-3-Clause (license)

Dependencies

archive, code_assets, crypto, ffi, flutter, hooks, http, jni, jni_flutter, meta, path_provider, web, yaml

More

Packages that depend on mediapipe_core