Voice Unlock

A Learnijoy app lock for students: during study time, a locked app opens only after the student reads a sentence aloud. One spoken sentence unlocks every locked app for a while (5–60 minutes). A parent PIN protects the settings and can unlock from the lock screen.

What's here

Folder	What it is
plugin/	@learnijoy/voice-unlock, the Capacitor plugin with everything native: the Android service that watches for locked apps, iOS Screen Time shields and extensions, the "Speak to Open" Shortcuts action, speech recognition and text-to-speech. Its API is plugin/src/definitions.ts.
mobile/	The standalone Capacitor app (React + Vite). All Voice Unlock code is in mobile/src/voice-unlock/: core/ is framework-free logic (sentence matching, study time, parent PIN, sentences), and VoiceUnlockApp is the root component.
app/	The original native Android app (Kotlin/Compose), kept as the reference until the Capacitor app replaces it.

The sentences the Capacitor app uses are in mobile/src/voice-unlock/core/sentences.json.

Build and run

cd plugin && npm install && npm run build   # the app uses plugin/dist
cd ../mobile && npm install
npm run dev        # in a browser, with the plugin's web stand-in
npm test           # core and screen tests
npm run build && npx cap sync
npx cap run android
open ios/App/App.xcodeproj   # Screen Time needs a real iPhone

In the browser, voiceUnlockDev.requestLock({ appName: 'Instagram', social: true }) fakes a locked app opening and voiceUnlockDev.speech({ type: 'result', alternatives: ['I like to read books'] }) fakes speech.

Native tests:

cd mobile/android && ./gradlew :learnijoy-voice-unlock:testDebugUnitTest
cd plugin && xcodebuild test -scheme LearnijoyVoiceUnlock-Package -destination 'platform=iOS Simulator,name=iPhone 17'

How locking works

Android (8.0+): a foreground service checks which app is in front every half second while the screen is on. When a locked app opens during study time, it brings Voice Unlock to the front on the lock screen. After a pass, the locked app shows again. Needs the microphone, usage access and "display over other apps".

iOS (16+), with Apple's Screen Time API:

• The parent allows Screen Time access and picks apps in Apple's app picker. Apple never tells the app which apps they are.
• During study time those apps show Apple's lock screen with Learnijoy branding. Its "Speak to unlock" button opens Voice Unlock directly on iOS 26.5+; on older versions it sends a notification to tap.
• After a spoken sentence the lock lifts for the chosen time, and a background extension puts it back.
• Optional, iOS 26+: a Shortcuts automation skips Apple's lock screen, names the app in the sentence, and returns to the app afterwards. For each locked app, in the Shortcuts app: Automation → + → App → choose the app → Is Opened → Run Immediately (turn off Notify When Run) → Next → add Speak to Open (Voice Unlock) → set App to the same app → Done. It doesn't open Voice Unlock while apps are unlocked, so it can't loop.

Before TestFlight or the App Store, Apple has to approve the Family Controls (Distribution) entitlement for the app and each extension's bundle ID. Development builds work with the team's automatic signing.

Not yet tested on a real iPhone: speech recognition (the Simulator has no speech assets), "Is Opened" automations, returning to Instagram and similar apps, and study periods starting and ending at their real times.

Moving into the Learnijoy app

1. Install the plugin in Learnijoy (npm install <path or git URL>/plugin, after building it).
2. Copy mobile/src/voice-unlock/ into Learnijoy's web code. The lock screen is part of VoiceUnlockApp, and it has to be listening whenever the app runs, so either keep it mounted app-wide or split the lock overlay out from the settings screens. The screens are React; core/ and the plugin work with any framework.
3. Android: minSdk 26. The plugin's manifest (permissions, service, boot receiver) merges on its own.
4. iOS: add the extensions, entitlements and Shortcuts action with mobile/ios/scripts/setup-voice-unlock.sh --project <Learnijoy>/ios/App/App.xcodeproj --team <TEAM ID> --app-group group.<bundle id> (safe to re-run), then request the Family Controls distribution entitlement for Learnijoy's bundle IDs.