I use Gemini, which supports PDF File uploads, combined with structured outputs to generate Course Sections, Levels & Question JSON.
When you upload a PDF, it first gets uploaded to a S3 Database directly from the Browser, which then sends the Filename and other data to the Server. The Server then downloads that Document from the S3 and sends it to Gemini, which then streams JSON back to the Browser. After that, the PDF is permanently deleted from the S3.
Data Privacy wise, I wouldn't upload anything sensitive since idk what Google does with PDFs uploaded to Gemini.
The Prompts are in English, so the output language is English as well.
However, I actually only tested it with German Lecture PDFs myself.
So, yes, it probably works with any language that Gemini supports.
Here is the Source Code for the core function for this feature:
js
export async function createLevelFromDocument(
{ docName, apiKey, numLevels, courseSectionTitle, courseSectionDescription }:
{ docName: string, apiKey: string, numLevels: number, courseSectionTitle: string, courseSectionDescription: string })
{
const hasCourseSection = courseSectionTitle.length > 0 && courseSectionDescription.length > 0;
// Step 1: Download the PDF and get a buffer from it
const blob = await downloadObject({ filename: docName, path: "/", bucketName: "documents" });
const arrayBuffer = await blob.arrayBuffer();
// Step 2: call the model and pass the PDF
//const openai = createOpenAI({ apiKey: apiKey });
const gooogle = createGoogleGenerativeAI({ apiKey: apiKey });
const courseSectionsPrompt = createLevelPrompt({ hasCourseSection, title: courseSectionTitle, description: courseSectionDescription });
const isPDF = docName.endsWith(".pdf");
const content: UserContent = [];
if(isPDF) {
content.push(pdfUserMessage(numLevels, courseSectionsPrompt) as any);
content.push(pdfAttatchment(arrayBuffer) as any);
} else {
const html = await blob.text();
content.push(htmlUserMessage(numLevels, courseSectionsPrompt, html) as any);
}
const result = await streamObject({
model: gooogle("gemini-1.5-flash"),
schema: multipleLevelSchema,
messages: [
{
role: "user",
content: content
}
]
})
return result;
}
The UI mostly works offline once loaded in due to aggressive caching.
Downloading Course Content was on the initial Roadmap but I removed it since I wasn't sure if anyone would like the feature.
Syncing stuff is a real pain in the ass but I'll implement it if at least a couple people want it.
Thanks for the suggestion, I’ll definitely try to make the app as language inclusive as possible!
Also, sorry if I might’ve been too vague with the post title.
The app is just similar to Duolingo in terms of structure and the idea, however it’s not specific to language learning but supposed to cater to any subject, really.
For example, I personally use it to study for my university subjects.
I'm making an App for Lemmy and I'm planning on adding that feature. I also want to make it so you only have to register once and the App can register you to all the instances you choose automatically.
Thanks :)